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ABSTRACT ■ 

The rationale^ developaent^ and structure o£ a high 
school oral coaaunication assessaent prograc are described' in this 
paper. Pollowing inforaation, on coapetency. based education and the 
need for developing and testing studenits* oral, coaaunication skills, 
the developaent of an assessaent -instruaent by tlie Glynn County 
(Georgia) school systea is diiscussed. This discussion reports on the 
selection^of priaary and alternate speaking tasks (priaary— speaking 
before a siaulated public, hearing, alternate— participating in a job 
interview!,: how raters were trained to evaluate speech perf oraances; 
how cutoff scores were established: what resources were needed to 
conduct the assessaent: and the effects of the use of the assessaent 
instruaent on student and staff . develcpaent during a pilot prograa. 
Appended aaterials include guidelines, for as, and records used by 
students and ti^achers during the pilot assessaent prograa. (fii.) 
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■'FORIwORD 

!n the spring of 1977 the .G'ynn, County School System was selected . 
as a>sptlpt site to assist in the Ident I ficat Ion .and develop comr 
fpd^nents needed for a cpmprehenslye cpmpeHricy program : 

In Georgia. After 3everaI^ months were spent col icctlhg Ideas from the 
porwnUnlty regard Tng what minlrnum cpmpetehcles should be ppssessed by 
graduatiBS of Glynn County hlqh schopls, high schppi courses were added , 
and revised to proy I de foi^ the deveioprnen;f or reinforcement of the 
.Identified 1 ite roje competencies. In additibn:^ work was beg.uh on 
developjng tests fpr deterrnining if students, had acquired^ the required 
competbncles. Since* an earlier :pubHcatipn outl ines the procedures used 
for construct I no paper-and-penciJ' tests, this document is limited to 
the rationale and .desert p+ion pf the .oral communication assessment prb- 
j^rartii This report is being reproduced and distributed wjth the hope 
that, it witl be helpful to other school systems donsiderihg local develop 
ment of simi I ar programs. 



Kerrhit Keenum,^ Superintendent 
Glynn County School- System 
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TIACH vAND i TOR ORAL GOMMUN I CATION SKI LIS? 

A wel!l known adage has. 1t that of al>l the; creatures .InhabftTng the 
Earth, f fsh are the least l ikely to ever dfscbver wafer. So It Is with 
speech commuhjcatlon. Speech comes, to us as part of pur Innate endow- 
ment as human, be! hps* We are engui-fed by speech communication in al l 
our dally affairs. Usually we are not directly aware of our oral communi- 
cation enyirbm-nent. But it is noriethejess vital to our wel l being and 
suryival.. . . , - 

Speaking and listening are prerequisite to success In school, f^st 
Instructions for classroom procedures arc delivered oral ly by teachers. 
Consequently, students with def icient listening ski his often exhibit 
errors which do hot reflect their mastery of subject matter, or they may - 
be wholly left out of classroom activities. Students who. listen poorly 
will likewise fall to absorb much of the materlaLto which they are exr 
pcsed. • Speech .performance also affect^ academic achievement,, StudentsJ 
who cannot/adequately express their knowledge are judged Ignorant. Some 
speech styles trigger stereotyped expectations of poor. ability — dxpecta- 
t Ions Which are li^kely to be selfrfulf M l Irig (Wi l l lams. Whitehead and 
Miller, j972). Quiet children may be appreciated for their "good behavior 
but they pre :subject to simi larly negative ^expectation is (McCroskey and 
Daly,^ 1976); Students who cannot adequately ask for assistance: wl 1 1 not 
receive adequate .assistance. One research study, for example, -found that 
reticent students progressed slowly through a self-paced reading program, 
despite normal' levels of reading aptltujlle. the fc*>^planatlon for thi>s' de- 
pressed reading perfpniiance Is that these students ra re l^y approached 
tMchers for Individual Ized help (Scott, Yates and Wheel ess, I975). 



Beyond the confines of schools, oraj communication proficiency 
.contributes to social adjustmeht and satisfying Tnterpersonal relation- , 
ships.. Youngsters with poor communication sklJIs are vleWed as unattracr 
. tlve by^ their peers and enjoy few friendship bonds (Hurt *and Prless, 
1978).* Antl -social and violent behavloc Is fr^equently attributable tp 
underdeveloped social sensitivity and lack of conflict resolution tech- 
niques; remediation programs have reduced the Incidence of aritl-soclal 
acts by means of; communication training (Chandler, 1973). Counsel lors 
acknowledge '^hat many family problems are caused by poor communication, 
and may be amel lorafeav by Improving Interaction between family members 
(^prqla Department of( HQman Resources, 1977). 

Speaking -ahd 'H-t^t^nlng arjs^^^^ less crucial In the marketplace. 
Corrimunlcation skills rank high among lists of managerial competencies. 
An\off leer of one computer firm, for example, states that his company 
p're^^'ers to conduct Its own training ;ln programming, but seeks employees 
nth strpng communication abl litres: (Gruner, Logue^ Freshley and Hus'efnan, 
J977); Profess lona I s-rdoctorsi lawyers, engineers, teachers—requl re^more 
than just -subject -matter expertise, they must listen effect! ye ly to their _ 
patients, clients or students In prder to Identify and analyze problems. 
They must :speak effectively In order to Implement their solutions: Indivi- 
duals who speak In a npnstandard fashion (Labov, 1972) or who withdraw 
from speaking (Richmond, 1976), will be regarded by personnel officers 
as prosepcts for pnly low status, low^r-iylng jobs. 'Even unskilled workers,, 
however, have occasion to engage In. job related speech. Including a sur- 
' prising amount of publ ic 'spenkln,g ( Kendal 1 , 1974). 

Speech curricula have tradltjonaiMy stressed the Importance of 
communication for the preservation of a demqcratlc society. Throughout Its. 



history America has vigorously founht to safepuard freedom of expression 
under the assumption that ful l citizen .participation ls\he surest 
guarantee apalnst tyrrany* Surely' not every citizen will deliberate as 
, a member pf^a legislative body, but numerous opportunities for citizen ' 
input . are available. These Include participation In civic associations, 
public hearlntjs, and citizen lofebylhg, especially at local levels* At 
very l,east, citizens are responsible for staying Informed, and much of 
the pertinent Iriformbtlon Is to be culled by listening. 

Finally, oral communication Is essential to full psychological 
development* We learn about ourselves, acquire a self concept, through 
Intei^actlon with others (Mead, 1934); Self-actual Izationj a sense of 
/fulflilnent (Maslow, I954)> usual ly entails Interpersonal activities, 
making contributions, exerting InfKuenSff', or being recognized In a 
social manner* In addition, speech can be used .for purposes of artis- . 
tic expression or self-discovery* ^ ^ 

The fact that speech communlcatlpn Is a naturally deveJoping con- 
stellation of skills does not Imply^that al I Individuals are effective 
communicators* The reader has only, to recollect his or her latest family 
argument or professional set-back to recognize the common need^fpr Im- 
proved communication Gkllls. Educators pccaslonally comment, "My students 
don*t need to learn how to talk. That-s one thing they can do — too much^ 
of*" But effective communication must be cultivated. Students may lack 
clarity In their speech* Their listening comprehension may not attain' 
Its fullest potential. Students who i^ommunlcate well In f ami 1 1 ar settings 
may lack the conf idence and flexlblll-fy needed to express themselves 
etfectlyely In a wider range of situations'* In contrast to the t^acher^s 
lament ^ hat "sttidents know how to t^slk all too well, effective comKjn I* 
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cation requires Judgbrnent--Judnement about selecting appropriate;! y 
adap^ted language, Judgement'about devising organizational patterns. 
Judgement about when and hov^ to listen. 

:Speaklng and iTstenlng, then,, are W-tal .^ Moreove^^^, feducators 
cannot rely on haphazard, ungutded learning outs <de pf the classroom 
to Impart, communication effectiveness. Stl 1 1,, of all the basic skills, 
speaking and l istening -are most often neglected In schools, this ne- 
glect transpires despite curriculum documents which urge attention to 
oral abilities (Georgia Department of Education, 1968)^ Undoubtedly 
a host of factors, discourage, teachers from Implementing oral communi- 
cation Instruction; The myriad of forms and tests constitute the evi- 
dence against which ..public schcv) teachers, administrators, and systems . 
are Judged. Teachers arc Ho id accountable for students* reading achieve- 
ment, for performance on mandated grammar tests,, for monitoring attendance, 
for giving enough homework, for not giving too much homework. But 
teachers are genera My not held accountable for teaching students to 
speak and .listen effectively. Consequently, little concerted Instruc- 
tion In speech communication takes place.- ^ . • 

J f students* speaking and listening proficiency were systematically 
evaluated, however,. It Is likely that schools would systematical !y 
fmpTement oral communication Instruction, that Is, one^^substantlaT 
benefit of laj:ge scale assessment of oral- communication skills lis that j 
such testing can guide In'^ovatlon In this curriculum domain. Indeed, 
exf)9rlence In Great Britain. and elsewhere demonstrates tjhat speech assess- 
ment has a "washback" effect on the amount and kinds of speech teaching 
undertaken In classrooms (Barnes, 1980). 

Another benefit of oral communication assessment Is that test results 

10 
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can be used to make decisions. about the best manner to place IndivTdual 
students in Instructfonal sequences. Assessment "procedures which yield 
fJrie?gralned analyses, rather than" global judgements, can be used for 
diagnostic purposes (Rubin, I9CI). Thus, for example, students who have 
difficulty in vocal production factors might concentrate on oral reading, 
while those whose djfflcultJes Me in the .area of organization might 
cycle' through a set" of story telling exercises befbre progressing to 
explanatory discourse. Students who demonstrate strengths In, say, literal 
comprehension o^ spoken materials might advance to Instructional units 
emphasizing critical I istening ski I Is, 

Speaking and I Istenlng tests can also provide valuable information 
for program evaluation • Since large scale programs of oral commLnicatlon 
Improvement are In their Infancy,- it Is especially important to evalu- 
ate their effectiveness, and to secure data which will enable these pro- 
grarps to be -'fine tuned,*' Program (and teacher) effectiveness Is best 
Judged with reference to student achl.evement on program objectives. If 
students are. hot achieving criterion performance levels In language use, 
for example, teachers and administrators wlil recognize that addltlpnal 
Instructional effort needs to be directed to this area. It Is. worth while 
noting, however, that student achievement can be Interpreted as an Indi- 
cator of program success only when student aptitude and Institutional 
resources — the raw materials with which the program has to work — are also 
taken into account. Also, student achievement Is not the only data which 
might contribute to/program evaluation, Attltudlnal outcomes, self- and 
pepr-eval uatlpns are also useful information for this purpose, 

A final use for speaking and listening assessments Is to certify 
students as having attained (or not attained) mastery In oral- common I- 



cation* Competency cer+lfltatloh In basic skills Is Increasingly demanded 
by competency-based e'ducatlon^ TO ^Promotion or graduation dect- 

sions may be based upon *suchcertlf I cation • As described In the foUow- 
I ng section,, the Stat^ of 6€K)rgra Is.ampng se^ system's natlpnwise 
which require tt*at pj-bflclency In oral: communication be certified for 
ai r high schppj graduates (Back! und, 1931 ^ 

< 

WHAT tSXRE? 

Although a majority ot the states In this nation report to be In 
..various stages of implementing something In their schools they cal 1 
competency-based ceducatidn, there appears to be several definitions of 
CBE and as many ideas as to how it should be Implemented (Schenck, 1978). 
A quick check in a dictionary of terms such as "comoetence/' ''competency," . 
and "competent'* provided, some clue to the main thrust of/Cf?E, ^however* 
Definitions "included: '"sufficient .for the necessities of life'*; "having 
requisite abilities of qualities"; "legally qualified or capable"; and 
"t>ie quality or atate of being functiphariy adequate^or having sufficient 
know I edgrr, judgement, skill or strength" (Webstiar, 1971 )• If the pur- 
pose of CBE Is. to produc^^ high school graduates who are, at least ^^"funcr 
tlonally adequate," how wjli the decision be made regarding what is ade- 
quate and how Jt wl 11 be verified?. Some states' answer t^ these questions 
is to leave this decision to commercial test publ Ishers who already have 
achievement tests prepared. By using these tests the knowledge tested- 
automatical ty becomes "what Is sufficient" and the Instrument Itself be- 
comes the means of verif ication. These states general I y have a minlfnum 
competency testing program rather than a comprehensive C3E program as 
it Is defined in Georgia and other states* 

In order to make an Impact CBE should be Implemented K-12 *rlth the focus 
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going beyond the mlnfmum graduation, competencies** Expected learner put- 
comes should be clearly defined and made known to' the? students In all 
•subjects and gr'sd^ levels. Some writers sec the^^CBE In struct I or/a I pro- 
cess as developing and c6mmunlcaj:f:ng objectj.yes/'dlagnosihg^ studg^^^ 
needs In relation to these objectives, measuring students' performance 

' . _ ' . ' ^ V ' ' ' k ^ 

against the objectives, and either cer^tifylng attainment If objectives 
are .satisfied, or beglnnJng the cycle over again with the. diagnosing of 
needsr^CGoldhammer an5. Weltzel , • I98l ) 

^Althouqh the.key to CBE appears to be the abi lity to apply designate 
ski I ls;Ctn€^^^ of skills receiving emphasll^ may vary:+o Include basic 
schobi skU Is, skill s'^f or sped f Id^'subject areas> skills .related^to 



adult /roles,* and qenerlc Je^^irnlnq/tKinklnq skll Is. Other variables in- 
elude: type., tJn.6^, and^%> laces for jj^^tructlon; ways to check learner . 

performance sdiqh as pa^er-and-penci.l t^sfs, school pr6ducts> simula- 

" ' ^ . 7'.' ' ^ . 

tlons;, andJi)ef;^oi?mahce; and ways, to certify competency attainment^ that 
Include, school personnel, jand perhaps, community experts (GoJ dhammer and 
Weltj^el,: r<)8.l). ■ . , • 



CBE IN GEORGIA \ /-'^-fi-Y. ■ .^^7 



GBE. was set In motion by a Georgia Board of Education policy In Nov 
ember, 197^.. Much of the terminology used in thls'pollcy was taken , 
.from the Oregon approach to competency-based education ,,:partlcujarly 
emphasis on, and description of, "life role skills." Rather than 
limiting CBE to^^the basic ski IJs* of reading, writing and. arithmetic, 
the ppllcy^s reference to life role, or adult role, skllJs Included 



health .and safety, citizenship, producer, and -'consumer sljll^Js. This, 
policy wt^, at le|ist In part. In response to the cry 'that the value of 
the high school diploma had depreciated. o . 



:fn the spring of 1977 school systems from nine of the ten Congres- '\ 
sibnal Olstrfcts agreed to serve as pilot systems for developing and test- 
ing strategies needed to Implement this policy state-wide. ^Thp districts 
of Da I ton City, Fulton County, Glynn County, Gwinnett County, Henry County, 
Latirens County, Muscogee County, Newton County, and Thomas County received 
state grants to aid with expenses of conducting pilot efforts for three- 
yearns;. Although each'scffqol system had special interest In particular 
facets of the study, all systems eventually addressed the problems of 
curriculum and Instruction, evaluation of competency attainment, reme- 
d ! at tbi^* guidance and advisement, record-rkeeplng and reporting^ and 
-speclaUeducatlon==?a.UJn. the^context of CBE. 

After the three year pilot effort the Georgia Board of Education 
ejected, to phrase In CBE Implementation over a period of -Several yo;.*s. 
The new course requirements, for example, were to become effective state- 
wide with the graduating class of 1984, while the basic skills require- 
ment would be needed by the class of 1985, .and the class of 1986. would 
need all competency requlrements^(basic skills and life role). 

Some of the pilot systems opted to continue with their progress, 
"to keep the morjentum going," and to not watt for the state-wide Vmple- 
mehtation schedule* Such a system was the Glynn County School System. 

HISTORY OH GLYNh^ COUNTY VS. CBE^ PROJECT 

Located fn the <:oastal region of Georgia^ Glynn County's population 
of 59,000 Is distributed over an aVea of 439 square miles and Includes 
Brunswick, St* Simons Island, Sea Island, and Jekyll Island. While ' 
tourism Is a major source of Income for the county, approximately seventy 
manufacturing and processing ,estab 1 1 shments produce naval stores and 
chemicals, paint and varnish. Industrial boilers, creosoted timbers, 
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,pulp saws ancf machinery, lumber products, fresh and frozen aeafoods, 
ship Impairs, garments, tools pnd dies, and othor products^ 

The Glynn County School System serves over 10,000 students K- 1 2 
with eight elementary schools, three middle schools, and two compre- 
henst'Ve high schools. 

In the spring of 1977 the Glynn County School •System was selected 
as the CDE pilot system for the First Congressional District and subse- . 
quentty received a three year state grant to Identify and develop 
components needed In a CBE program. The first major task undertaken was 
to Identify minimum life role skills or competencies needed by high 
school graduates In the broad areas described in the State Ftoard Policy. 
The original policy stated, for example, that ''each citizen should have 
proficiency in reading,, writing, listening, analyzing and speaking." 
Several month^^were spent gathering input from teacherSi students and 
other local citizens regarding" what minimum competencies should be 
possessed by graduates of Glynn County high schools. 

The decision was made during the first year of the pilot effort to 
use -paper-and-penci I tests, for several . of the IP/e role areas and that 
development should begin Immediately. There was never any questions, 
however, .that the assessment .for writing and speaking competencies should 
be performance based, although* development of these tests was delayed 
pending progress, reports from the State Department of Education. A re- 
port of the procedures, used In developing the paper-and-pencl I competency 
tests Is available In a separate document (Yeany, Okey and Bazzle, I980). 

Another task completed was to review course guides to Insure that 
opportunities were provided for learning the^ life rote competencies. 
This task was made easier by the existence of current course guides that 
Included performance based objectives* 



.New courses In career planning, health and safety, personal finance, 
citizenship, government, principles of economlcs/buslness/f ree enterprise, 
remedial reading, and remedial math were added to the curriculum. 

An Individualized student advisement system was Initiated using 
cross-graded groups of students that were to remain with the same 
teacher-advisor while In high school* Advisement wo^uld be primarily { 
of the academic type for the first few years. 

After two years In the project,. It became evident that It would be 
some time before the State Department of Education could provide state- 
wide guidelines for writing,' speaking and listening skill assessment, 
so development of ways to verify .competence In these areas was begun. 

As consideration was given to the Identification of oral communi- 
cation competencies and Indicators that would be used as evidence that 
"each-cltlzen has prof Iciency In speaking," .the emphasis on' life role ^ 
application was continued. It was decided that' opportunities that, 
citizens have for oral communication would first be Identified and that 
a skills analysis would reveal the skills needed to successful I y complete 
the tasks. A district CBE planning committee generated a prellml;nary 
?lst of sltu2itlon55 and skills which Included, giving and understanding 
directions, participating In employnoent Interviews, conversing on the 
telephone, and participating In various citizenship roles. 

THE ASSESSMENT TASK • ■ ' . 

After the planning committee completed Its Initial Identification 
of. speaking and listening competenoles, several alternative evaluation ^ 
procedures were drafted. This preliminary proposal appears, in Appendix 
A* The* communication tasks which were proposed attempt to create con- 



text for talk which have *Mffo role*' significance (I.e., employment, 
peer relations, consumer affairs). They provide Students with moti- 
vation for communication beyond the avowedly evaluative purpose of the 
activities, and they attempt to minimize ^threat. Moreove^, the pro- 
posed tasks construct or simulate situations— audiences, goals, settings 
topics— so that students might practice adapting to situational con- 
straints. The District decided to concentrate on speaking assessment 
In this present phase of program developrr>ent, and to s.elect tasks which 
would not place the entire burden of testing on English faculties, 
tasks "Which would conform to curriculum strands in other departments* 
Consequently, the job Interview and the public hearing tasks were chosen 
for further development and pilot testing. 

The .Interview procedure, adapte'd from a previous research study 
(Rubin and Nelson, 1980), was derived from observations of numerous 
interviews at fast-food res'taurants. Students first*^ completed applj- 
catibn^forms In which they could specify th^t they were seeking work as 
a cash'ler,. host/hostefes., or cook at a fictional pancake house* (See 
Appendix B. ) The Interview .schedule, reproduced In Appendix C, con- 
sisted of about 25 open and closed questions about students' quallf?- 
cations, experiences, arid Interests. It also Included one question 
calling for a re.latlvely.extended narrative response. 

- After several revisions", the public hearing task presente'd a slmb- 
lated situation in which students were called to. testify before a fic- 
tional board of education, the board was considering three propositions 
of local student Interest. ^(The meeting agenda appears In Appendix D^) 

Each student chose to speak Iji support or In opposition of one of the 

. ^ * - ^ ^; • - 

proposals.' Studisnts addressed a panel of "board membeVs,'' actually 

throe students portraying that role. 



Several days. prior to' ad^Jnlsf^ratlon of both the Interview and • 

%\ 

the.:publlc hearing tasks, students received and discussed student guides 

i ' ' ' 

to oach of the communication sltuatl^ons. (See Appendices E and F. ) These 
guides acquainted, students with the Importance of these forms of communi- 
cation; The guides also explicitly enumerated the criteria along which 

^^s^tijdents would be evaluated* 

Thirty ninth-graders of various ability levels wore subjected to 
the employmeht Interview procedure. The Interviews were recorded on 
audid tape and evaluated by pairs of raters. Appendix G contains the ^ 
rating Instrument for this task. This procedure was found to be feasible, 
but more time consuming than the publ Ic hearing task. Also, raters 
experienced, difficulty In rendering Judgements because of the Interrupted, 
conversational nature of the resulting "student discourse. Consequently, 
the Interview task was designated* as an alternate form 6f the speaking 

^.assessment, to be rsed ior those students who require retests after first 

taking, the public hearing assessment. Data gathered from tlje Interview 

' ' ^ ... . --^ 

task' appear elsewhere In this report. » ' . . 

the public heaVlng task' was^selected as* the primary assessment 
procedure for several reasons. . It demaridjs continuous discourse, i*hlch 
was found to ease the Job of using rating scales. ' The task creates a 
situation In which students can express their Interest and aliso draw^ upon 
commonly held knowledge. $tlll, the. agenda, Is presented In a manner 
which suggests some subject matter (reasons to support or^ not support 
the proposition) for those students who need prodS to Invent their own - 
content. The communication context simulated a "life role" setting In 
•which speech Is used for civic purposes. Thus' Instruction In this 
language art. can , be shared wltfi the social stud^les faculty. The setting 



also' bhcxiura^^ fprrtal language, not as an arbitrary requirement, but . 
as .ajh^tura! function of renf stiBr shW 1975; Rubin, 1979) 

In the. face of social distance* RnaJly;, the public hearing task proved 
toi^be logistic?! My simpler than the bne-tp-one Interaction of the e/nploy^ 
meht; Interview* • ' * 

Complete Instructions for admin I steering the publle hearing assess- 
>meht task ^re presenfed^lri Appendix G. In .brief, the test administra- 
tor begins by reminding the students of the Importance of airing citizens 
>y4ews, and attempts to alleWate students^ speech anxiety. The adminis- 
trator ireads the aglSnda^ albud In order to minimize contaminating effects, 
of reading ability. Students prepare their remarks for about five min- 
utes^ and are .permitted to use brief notes. Students are called to 
speak "In a random order. ThcTy stand at a podium Inset Into the audience 

* * 2, 

i 

and address a **Board** composed of three^l unteer students. If. a video 
camera Is usfjd. If Is set, at an. angle to the speaker producing a half 
profj M> shot. Students speak for a' maxlmutn of three minutes each, ^ 
Experience with students who had receljved no prior Instruction revealed 
that 'the majority spoke for 90 seconds or less. 

SCORING THE PUBLIC HEARING'tASK S ^ 

^^fhe r^atlng lhstrui3f)ent devised, and^several tlrrtes revised, for this ^ 
:prpject' Is a rhfetorjcal trait. mesrsura (Rubin, 1981). The crlterla'and 
quality Indicators for rhetorical trait Instruments correspond to. the 
particular rhetorical demands- of asfsoclated communlcatlpn tasks, "^here- 
fpre^'djstJnct rating scal^es must be cfeve loped for each assessment task. 
This approach Is based on the premise that there are no absolute stand- 
•ards 6f communication quaMty, .but^ character I sties., of ef fecMve com-r 



muntcatton are situation-specific (WIemann and Backlund, 1980; Lloyd-Jones 
I977)* the original dra^t bf the public speaking rating form Is pre- 
sented In Appendix H* The final version, presented In Appendix I, was 
sifghtry abbreviated to facilitate rating. (See Appendix J for the 
employment Interview rating lnstr;;;nent.y 

One type of rating scale considered for use In this project, but * 
rejected. Is the general Impression or global scale. Raters using general 
impression scales render a single overal I Judgement of student performance 
Such scales carf be used reliably and rapidly (Cooper, 1977), but yield 
only gross Information. Rhetorical trait scaleis. In. contrast, consume 
more time and .require extensive training. The advantage of rhetorical 
trait Instruments Is that the evaluative criteria and fhelr descriptors 
are eijcpressed explicitly. While each student receives a single score 
-which Is, the. sum of the ratings on each criterion, the meanlngTof that 
^single score ^Is evident. Rhetorical trait scales are especial ly advanta- 
geous In guiding Individualized diagnosis and remediation, and In guiding 
currlcular Innovation In general . ' . V 

fij qrder to aid readers who may wish to adopt this, or similar, 

' V 
rating Instruments, the following section exp 1 1 cates" the evaluation and 

' * * ■* " ' # 

their quality Indicators. • - ' " 

Criterion #1: Introduction - ' , 

^(1) none - Student makes no effort to preface remarkis 

with attentlon-gettl*ng or orienting statement. 
(2) Just names proposal - e.g., "Pm gonna*, talk on letting 
students drive to school.'' 
^ /3) names p/x)posal and Introdu'jes self - e.g., 

• ' •'My name Is Brett Lucas and I'm a sophomore at Brunswick 



High; l\want to talk about why you -should 'at stu- 
- dents drive to school 
(4) name? proposal and attempts to capture Interest - e.g.,, 

"A high school' that neglects +o teach mathematics would 
. be called.a bad school. In the same way, a school which 

doesn^t teach responsibility Is a bad school. That's why 

I want to talk against the plan to keep students from 

driving to school." 

Criterion #2: Position . . 

(1) no point of view - descriptive rather than persuasive, 
e.g.., "If students drive to school 1t wastes a lot of 

to 

gSfi and a lot of students fool around and get Into 
" trduble v(lth their cars. If students can^t drlvft to 
schOTJ a lot of them tan^t get to theTr jobs or some^ 
havei/to travel a long time on the ^)us."" c 

(2) • distorted p6lnt of view as Ft re I ares to^the proposal 

shows confusion or strays from, the "point, e.g.,, "I think 
you 'should let students dri ve ., to' school • There^s a 
lot of other stuff students shouf J be aUoy^d to do In 
school; Llke especlal ly It's not fair we caiyt smoke." 

(3) -^yague point of view - weak !n stating^ position, e^^g*, 

"Something should be done about students driving to . 
' school ; It's ^ problem that needs to be dealt ;Wlth.." 

(4) states opinion forcefully or with situational qualifier - 
e.g. , ."Students should .nbt^'bo permitted to drive to school 
and this rule- should be strictly enforcedi" Or, "Only 
those students^ho can show they need cars to get fo wpi^k 
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- should be allowed to drive to school. Only those few 

exceptions should be, pennftted." 

I 

Criterjoh #3: Reasons 

('I) unsupported assertion - provides no meaningful Justifi- 
cation for point of view i e*g., "I Just th I nK students 
should be allowed to drive, to school , 7cause that wouldn't 
be too good If they couldn't/! > 

(2) uneiabbrated reasons given - sta-^es one or more reasons 
In support of position, bur reasons are not developed, 
e.g., -'If stiidebtsjcan't drive to school they'd probably 
Just park a few blocks away,' and Jt would probably cost • 

- more to' run the buses anyway." 

(3) . at least onia reason su'iported - e.g., "A lot of students 

are used to driving ta school and It's Important to them, 
they would find a way to drl^e even If they couldn't 
park on the streots^arourid the school. Then the streets 
would get rea^l busy and the people who live there couldn't 
find anyplace to park and they'd get al.l angry, ^d It 
wouldn't solve any problem anyway." . 

(4) , more tjian one reasc.2 supported or especially apt support - 
Includes arguments especially adapted tb the^^^ftrspeqtlve 
of the Board of E^ucatlo^, e.g., "As memb^s of the Board • 

• of '^Education you are all concerned that students, get the 
best education: I'm sure you would agree that part, of a 
teen-ager's education Is learning hdw to be' responsible. 
You are all. responsible members of this community and 
^ I'm sure that you learned this sense^^bf responsbl llty^ 
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by being given responslbl I Itlos to take care of, not 
by having them taken away. I think that teenagers today 
learti responsibility by having cars* They have to make 
suro>theyVe fixed, work to earn money for gas, and obey 
good rules of driving. If you take away the right to 
drive Ho schcxi youMI actually be taking away from stu- • 
• 'dents' education. ^ 

Criterion #4: Organlzatipn 

(1) Ideas wholly unrelated - e.g., ''Some students are very 
good drivers. Mot every student has a car. For a lot of 
people, the buses, Just take +00 much t^me•" ^ 

(2) Ideas only Implicitly related (or only one Idoa e*xprnssod 
no oroatilzatlonal pattern possible^) 7 o.q., "Most students 
have Jobs, They need tholr cars. l;t;;..would mean a lot 

• of trouble for students If they couldn't drllve to school 
and some of them might even lose their Jobs." 

(3) logical sequence or simple transitions e.g.,,^'Most 
students have Jobs after school. They need to^drlve to 
school, because there's rto other way for them to get to . 
their work." Or, "Another reason why students should 
drive to school Is . . ." 

(4) proper emphasis and exp I Iclt connections between Ideas - 
p*g., "There are several reasons why students should be 
permitted to drive to school. First, many students need 
their cars afs the only way to reach after school Jobs. 
Another reason Is t^^t^he bus ride tis^oo long for a 
lot of students,. But the most Important reasonf why stu- 
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dents shoufd be allowed to drive to school Is that 
this teachos sdnse of responsibility. Owning a car 
Is a big responsibility. You have to obey traffic rules, 
pay- for: gas, Tceep track of oi l changes,^ .and so on. If 
students can't drive to school , It's like the school say- 
ing we can't' be responsible. I don't think the school 
should fake away this chance for students to learn to 
take care, of Important responst.bl 1 1 ties. 

Criterion #5: Gohcluslon 

(1) no conclusion or merely stater, that remarks are finished 
e.g., "Wen,. I guess that's It.'* 

(2) Just thanks Board members or Just restates position ^ ^ ^ 
e.g.,, "So that's w! ; I think you should allow students * 
to drive to school.'* 

(3) restates position and thanks Board members - e.g., "So 
that's why I think you. should allow student to drive 
to schooK Thank you for giving me this chance to ex-^ 

. press my views. i ^ \ 

(4) summarizes pos^^lon and of fer^ thanks - e.g., "In con- 
elusion, these' are the three main reasons why I hope 
you iiilll continue to let stad^nts drive to school: 

♦ 

Students need their cars because most everybody works. 
It wilt save money oh the^ buses. And students work 
hard for the right to have their cars* Thank you very 

much for giving me thls'tlme to speak to you." 

* ' • 

Criterion #6: Language Styljs 

(i) Incomprehensible or reads prepared statements - speaks 

ERlC . 24 . . 



with pronunciation, vocabulary, or grammar which cannot 
be understood without great effort, or attempts to avoid 
the challenge of extemporaneous speech by preparing a 
written text. 

(2) slang or Inappropriate language used - speaks with 
enurtclatlon., vocabulary, or grammar which creates an over- 
all Impression Hi fitting this consultative/formal situa- 
tion. 

(3) minimally fluent, appropriate formality - speaks with nor- 
mal dysfluencles and "fillers" and with language suitable 
for use In creating e positive Impression ^on high status 
Individuals. 

(4) vivid phrasing, highly comprehensible- e.g., "Ladles and 
gentlemen of the Board of Education,' In serving our county 
on this Important panel, your dej Iberarf©^ have a welcihty 
Impact on the future of all citizens. In rendering your 
decisions you must carefufly balance the potential costs 
of those decisions against their probably benefits. And' 
.you are aware, I'm' sure', that those benefits and costs 
extend beyond the schools themselves to every facet of 
our ^community's economy;" also Includes use of figurative 
language and analogy, e.g., "Depriving students of the 
right to drive to school because of traffic problems 
would.be like forbidding homeowners from Installing air 
conditioners because they sometimes put a strain on the 
elecfrlcal suppfy. Cars and.alr co>^dttloners are both 

^ luxuries In a way. But they have become part of our life 
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Styled. ' In both cases', people would find a way to get 
around the rules In order to keep the llfe'sfyle thjv 
are used to. Instead of bannlng^a^ you 
would probably find a way to Increase the supply of 
electricity* Instead of banning cars, there must be 



another way of solving the traffic problem.** 
Criterion #7: Vocal Delivery 

(1) monotone or Inaudible - speaks with "mechanical" tone . 
of voice, or cannot be heard. 

(2) distracting tone or rate - speaks too slowly or too 
fast, or with" odd inflection (such as rising Inflection 
at end of declarative sentence). 

(3) conversational tone and rate - speaks In a manner which 
' \ does not ca 1 1, attention to Itself and does not strain • 

the listener. 

(4) emphatic tone, varied rate— conveys authenticity and* 
conviction by appropriate pausing, voice modulation, 
and stress. . 

Criterion #8: Gestures 

(1) distracting rwrnrorlsi^s or posture - Includes drumming 
on podium, excessive preentnn^ overly casual as well 
as overly dramatic gesturing stance. 

(2) no eye contact with Board members - consistently fixes 
gaze on notes or to side. 

(3) eye contact est^bl!lshed|. comfortable posture - estab- 
lishes eye contact at least sporadically, stance relaxed 
norma I movement. 
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(4) ' extended eye contact an dr some' appropriate gesturing 
' / , ' ' luses hand or ^facTal- gestures to IMustrate or reinforce 
language/ relies only minimal IV on notes, 
' A score of "0** Is asslcined for nonperformance ♦ A total score Is ob- - 
talned by summing the ratlnqs of aH eight crlte1-la. - The potential range 
.of scores In thus 0-32. 

TRAINING RATERS TO EVALUATE SPEECH PERFORMANCES 

A student's score on the oral communication assessment Is, of course, 
supposed to reflect that student's "true" communication abll'lty. In order 
for the score to be Interpreted meaningfully In this manner, the score ^' 
ought not be greatly affected by extraneous factors such as the time of 
day at which the speech assessment was administered, the other students pre- 
sent In the audience, or the agenda Item about which the student chooses 
to speak. We presume that unless some events (e.g.'. Instruction) Inter- 
vene to alter the student^.s communication ability, that student would ob- ^ 
tain a similar score on another administration of the assessment. That 
Is, we desire that the examination yleVd a r el lable Index of the student^s- 

speaklng abl I Ity. ^ • 

Judgements of quality, such as those called for by the rating Instru- 
ment used here, are prone to subjectivity, likely to be affected by Indivi- 
dual raters* Idiosyncratic tastes, moods, and previous experiences. T'here--^ 
fore disagreement among raters Is a major source of unreliability In per- 
formance, scores ♦ The scores are contaminated by error to the degree that 
they fluctuate because of Idiosyncratic differences among raters, rather 
than .because of abP Ity' dl fferences among speakers. It Is essential tlift , 
r«iters receive adequate training In order .to ensure Inter-rater rellabll.lty. 
Z ' Previous experience with rating programs of this type suggests a 



number of design features whf.ch enhance Inter-rater r?t lability (Braddock, 
Ljqyd-Johes and Shuer, 1966; Ryblh, 1981). Agreemeht among judges^ ts 
facilitated by selecting raters with similar backgrounds, by Including 
raters. In the process of constructing -the rating Instrument, by assign- 
Inpraters to students of whom they have no prior expectations, by pro- 
V^Wlng raters with opportunities^ to establish "anchor points" In common, 
and by allowing raters periodic gpportunltles to "recalibrate" themselves 
byrcomparlng their scores with those of their colleagues. These features 

. wiere Implemented In the presisnt project. .j 

y 'Rater training began with an Intensive workshop on speech evaluation. 
The first half-day emphasized the topics of communication apprehension and 

• stagef rlqht, language variation and nonstandard usage, and the role of 
speech situation or context in determining effective style. The renalnder 
of the day introduced raters to the" job Interview and public hearing 
assessment tasks. Including exposure to samples of student performances 
obtained Irrpllot trials. Raters also viewed and discussed prototype 
ratl'hg Instruments. 

At a* second training session raters evaluated sample student^^ speeches, 
discussed evaluation criteria In depth, a^d deliberated over alternative 
forms of the rating instrument. Detailed procedures for administering 
the employment Interview and public hearing tasks were also presented. 
Following this session the two tasks were administered to. thirty ninth- 
graders. Raters scored these performances, and Inter-rater reliabilities 
were computed. 
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Results of this field test were presented^ at a third meeting. The 
.decision was made to use the public hearing task as the primary assess- 
ment. Administration procedures and the rating Instrument were further 
refined. Following this session, public hearing speech samples were 
collected from approximately 150 ninth-graders.^ Ten raters^were then " 
gathered in a rating session simulating, for the first time, actual 
testing conditions. The session began with a review of the criteria and 
the quality level descriptors. Twenty videotaped speech samples were 
rated In common by all judges. The ratings on each criterion were shared 
publicly and discussed. Raters rendering scores diverging from consensus 
were encouraaed to brlnn their standards Into comformity. For the re- 
malnder of the rating session, judges worked in. pairs, comparing total 
scores aiter viewing each student^ s speech. When scores within a pair 
diverged significantly, p third "floating*', rater was called in to recon- 
cile the disagreement. (See following section on logistics of rating.) 
At a final meeting additional raters were trained, slight adjustment^ to 

V 

the procedures were made, and plans were drawn for assessing the entire 
ninth-grade population of the dlstr-li^rt. 

LOGISTICS OF RATING SPEECH PFRFORMANCES 

Two procedures for rating student speeches have been tested. In the 
first method, students are videotaped as they speak, and these videotapes 
are later viewed by pairs of raters. Using videotapes In this manner re- 
sulted In strong Inter-rater reliability, but was quite time consuming. 
In the second method, raters evaluated student speeches "live." The 
speeches are audio-taped so that a third rater can review them If the ori- 
ginal pair of raters fails to attain satisfactory agreement.. The "live" 
rating procedure resulted In somewhat lower, but still adequate. Inter-rater 
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* • relJablUlty, and was^more efficient, 
, When raters work In conjunction with videotapes, the test admlnis- ^ 

y , tratpr must keep careful records of the contents of each tape, (See 

/Appendix K*.) Af the subsequent rating session raters are paired with 
I teachers from their own schools, the teams are assigned to evaluate •! 

I ' vfdeptapes of studervts from schools with which they are not familiar, 

- . • ^. • 

i, The rating team views each-sl-udent performance together, and may re- 

f / y^^^ ^ speech If necessary. Each rater enters a total score on a tabu- " 

latlon sheet (Appendix- J) after, completing the rating Instrument form, 
f^+^rs. niay not discuss a speech^ until they have scored It, nor may they 
change their scores after discussion. If the ratings of team members 
: differ such that one rater has given a score above the predetermined cut- 

1 off point and the other rater has scored the student below the cutoff 

1/ point, a. third '^floating" rater, reviews the videotape and marks a third 

rating form. That third total score Is also entered on the tabulation 
sheet. (A later section describes how cut-off scores are determined.) 
[ the "floater" Is not Immediately on hand, the^team will proceed to rate 

other students. -When the "floater" becomes available, he or she may skip 
• around on the videotape to rate several speeches for which the team has 
failed to reach agreement. Initial experience with rating videotaped 
Speech samples Indicates that the task requires .2 personnel hours per 
student. It Is likely that speed cduld be Ijncreased somewhat with 
^; additional practice. ^ 

^ When, the "live" rating method Is used, raters are again paired and 
I do not Judge students with whom they are familiar. The raters sit among 

the audience of. the public hearing. They do not, themselves, administer 
ft ttie communication task. Rather, -a third staff member acts as test 



adralnlstratbr, taking cues from the raters as to the rate with which 
to cal:) .students to the ppdlum. In order to minimize "down time" due 
to the raters' clerical tasks> thQ speaking order Is determined ahead 
of t\m* Each rater fs provided an ordered stack of rating forms onto 
which students' names, class, speaking order, and the date have already 
:been Entered. Raters do hot total their .scores Immediately. Total 
scores are figured and entered onto tabulation^ sheets after all students 
have f inished speaking and are dismissed. A. cassette tape recorder, 
external microphone fastened to^the podium, records all student per-r 
formances. A third rater uses this tape recording In case the tabulation 
sheet subsequently Indicates that the. "live" team has failed to reach 
agreement as to whether a student has attained the cut-off scpre, Inltla 
trials of the "live" rating method Indicate that about 7l5 personnel 
hours are required per student. 

ESTABLISHING ^ CUT-OFF SCORE 

The oral communication assessment is Intended to be a criterion 

referenced test. As opposed to. a norm referenced test. It does nbt 

seek maximum discrimination among students, but Is Instead designed to 

discriminate only between those who have mastered minimum levels of com- 

munlcatlon competence and those who have not. Items (I«e\, tasks and 

crTterta) were selected on the basis of their context, not because of 

their difficulty. Ideally, all students, at least after Instruction, 

would be characterized as masters by the test. Students who do not demon 

strate mastery on their first attempt have the opportunity to retake a 

different .form of the examination (I.e., the employment Interview task) 

> * 

without penalization. 

A cut'-off point discriminating between mastery and nonmastory was 



-established by a "comb I nation of lotjical and empirical analyses.. The 
rattn5; ^oFm was co'nsfrubted so that- a, quality level of "3" on each 
cHtepIoh generally represented adequate performance. The project staff, 
however, fe I t>hat students ought to be given latitude to pass the examl 
natlbn oven If they fal l to demonstrate adequate peribrmance ;bn some of 
the criteria. For the or Ig I na I. ten- I tern Instrument, therefore, logical 
anaj.ysrls suggested a cut-off score of 26. Students scoring 26 or better 
' Would be cert If fed as competent In oraLcommunlcatton. * 

/Table t presents results derived from a trFal Jnvolving 106 stu- 
dents at 3 ability levels' from 2^ schools. None of these- students had 
received prior Instruction In speech commuh.lcatlon. On the basts of 

TABLE I ' 

.Public Hearing Assessment Scores By Ability Level - 
Videotape Rating Method 

(Orlglnal^JO-ltem Rating Instrument)^ 







Low 




r 




Total ■ 


.Ability 


Average 


Advanced 




(N=I06) 


(N=20) 


(N=36) 


(N=50) 


mean • 


22.74 


19.80 


21.25 


24.98 


standard deviation 


4.53 


2.48 


3.33 


3.40 


mode 


20.00 


20.00. 


23.00 


25.00 


median 


- 22.00 


20.00 


21.00 


25.00 


achieving mastery {%) 


30.20 


5.00 


16.67 


50.00 



Potential range:' 0-40 . 

videotaped speech samples, about 30^ of the students obtained scores equal 
to or greater than 26. A second trial Involved 97 students from groups 
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""simHar to .those rated'Jn the first ariaiysjs. "These students had re- 
celysBd cspmp. fhstructlohal preparation, for the assessment • ;They were ^ 
■rated. "by the "live'' method. About 76^ of these students attained th.e^ , 
.ciit^f^f . score, as Indicated In Tablp 2. These results were deemed 
acceptable.- Extrapolating to the revised eight-Item rating fo;-m, a 

, — — *> . , , ' 

\ . . . ' * 

• ■ ^ - • . " ^ • - Table 2 •. • * ' - 

Public Hearing Assessment Scores. by Ability Level - ' 
"Live'- Rating ^tethod . 

(Revised lO-ltem Rating Instrument)' 



* 


. Total 
(N-97) 


■ Low 
Ability 
(N=25) 


Average 
. (N=33) 


Advanced 
(N=39) 


mean 


28.29 


25. 12 


27.91 


29.97 


standard deviation 


4.88 


4.55 


4.85^ 


3.76 


mode- 


27.00 , 


27.00 


■ 27.00 


28.00 


median 


28.00 


25.00 


28.00- 


30.00 


achieving mastery ($) 


76.30 


56.. 00 


75.76 


89.70 • 



Potential range: 0-40 
cut-off score of 1-9 was established. 

'rSY(H)METRIC ADEQUACY OF THE PUBLIC HEARING ASSESSMENT 

content validity. The content validity of a tost Is a function of 
the degree to which Its Items reflect the appropriate domain of skills 
and/or knowledqe. In a performance tost such as the public hearjng assess 
ment^ context validly refers to the selection of the task and the criteria 
for ©valuation. Any claim that tijis.examlnhtlon comprehensively samples 
the domain of speaking skills would overstate the case. A communication 



assessment with strong content vahdlt'y would need to Include several 
'tasks represent! nc] a wide range of. communicate^ sltiiatl.ons and func- 
tions. This, assessment procedure, even considered In conjunction with 
thevemployment Interview task, samples only from the more formal end 
rofrthe; spectrum of commun fcati oh. contexts ♦ 

Stfl l-, the publ ic hearing task possesses a degree of context 
yalld-lty because It conforms to the objectives specified by the school 
district staff: It represents a "life role" situation, calls for ex- 
i-erided discourse, and 'demands a standard- tike language style. The 
evaluation criteria contribute to content va| I dity because, like all 
r^torica.l trait Instruments, they were derived by means of a rhetorical 
tiask- analysis of the assignment. Indeed, the criteria do reflect the 
classical canons of rhetoric: Invention of subject matter, ^organization 
language style, delivery, and control over extended discourses Finally, 
the selection of public hearing agenda Items, the topics, . enhance con- 
tent validity Insofar as field tests Indicate that studenj^s find these, 
particular topics to be meaningful and manageable. Moreover, the topics, 
evaluation criteria, as well as the communication situation were all 
passed by the project staff. That staff constitutes a panel of experts, 
being comprised of experienced grade level teachers from English, social 
studies, and career education. 

criterion referenced validity. -. !f the results obtained with a new 
Instrument conform to those demonstrated either concurrently or at a 
future time by some other accepted measurement technique, then the new 
-Instrument is said to possess criterion referenced' va 1 1 dIty/ Two 
criteria were used In an attempt to validate the public assessment Instru 
ment: (|) teacher ratings of typical classroom communication and (2) 
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abiMty. level as Indicated by^placSmont In abli^lty level tracks^ 

'For each of the 106 students who had participated In the publ ic 

Jtk^ln<3 assessment, ^fwo teachers familiar with- that student rated 

the, student's typical cVassroom communlcat-}on-ski"l4s«-7-The -teachers 

used'a.form, prepenfed In Appendix L, which was developed for use In 

'the f^Msachusetts Assessment of Basic Skills (Massachusetts Department 

of Education, l'979)» The Inter-^rater reliability for those classroom 

communication assessments were r».6306. In order to ascertain the, 

validity of the publfc speaking assessment as referenced against this 

criterion, the correlation between average classroom rating and the 

average total public hearing rating was calculated. The resulting 

correlation coefficient was .0760™' 

Although the. public hoaring assessment Is a criterion referenced 

test, some Indication of the validity of total i ores can be devermlned 

by examining If these scores discriminate among students In a manner 

consistent with previous classifications of students^ overall academic? 

ability. The school district divides students Into low, average, and 

.adviiinced ability tracks on the basis of student sel f' selections, teacher 

recommendations, and standardized test scores^ ^^astery 'decisions (at 

two levels: mastery vs^. nonmastery) were cross-tabulated with student 

ability (at three levels) In order to determine the dependence of 

student performance on this examination with overall student ability. 

These cross-tabulations were perfo^rmed for two test administrations 

Involving 107 and 97 students, respectlveiyi (See Tables I and 2 for 

percentagns of students achieving mastery broken down by ability level,) 

2 

The resulting 2x3 contingency tables were .subjected to Ghl anaiysos. 

<? ^ 

Obtained values of ChK were 18*47 (2 df, p<'.OOI) for the videotaped 
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administration and 9.53 (2 df, p<.005) for the sample rated "I Ive." . 
In both cases the frequency of criterion !e»'e' performance was posi- 
tively related to abli Ity level. . ' 

internal . consistency ^. The scoring, procedure for the public hear- 
ing assessment creates a^total score by summing the raWngs on eac^ ^ 
criterion. One potential' source of error, or unreliability Is lack- of 
hoftpgenelty-^among^he crlter^la.^TRiif^rsTTf^^^^ Items do not all, 
contribute to a "pure" Index of corrmunlcatlon skill, then measure- 
ment error Is Introduced In summing the criteria* In order to ascer- 
tain the Internal consistency of the rating Inst rumertt, an Index of 
homogeneity, Cronbach's alpha,' was. calculated* Cronbach's alpha ranges 
from O-I.O, and can be Interpreted like a correlation coefficient* 
Two -administrations of the speech examination were each rated by two 
5ets of raters. The original lO-ltem Instrument was used to evaluate 
106 students by means of the videotape method. Internal consistency 
velues oblalned for thjs administration were .872 and .879. A revised 
10- Item' Instrument was used to evaluate 96 students by means of the 
n Ive" method. The resulting" values -for Internal consistency were 
•859 and .817.' While these values Indicate strong homogeneity for 
these 10- item scales, It should be noted that interna f consistency 
generally Increases with the nymber of Items. It Is possible, therefore, 
that the revised 8-ltem rating scale may display slightly lower homogeneity. 

The internal consistency of the employment sn-iferv lew rating scale 
was also examined. Thirty* students participated ih this alternate form 
of the speech assessment, and were rated by three sets of Judges. Obtained ' 

values of Cronbach^s alpha for the nine- I tern employment Interview rating 

• * ♦ 
•Instrument wore .917, .683, and .887. 
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^, ^ . yequlvalont forms roHabH tty^ Because the, public hearing assess- 
ment ts a criterion referenced test, It Is necessary to devise alternate 
forms of the examination which .can be administered to' those who do not 
•.'^ pass It InltlaNy* The alternate form must be equivalent to the primary 
' assessment procedure, at least In the sense- that performance on one i 
must be^ positively related to performance on the other. In order to 
detennlne If tRe employment I nterv I ew cou^^^seVs^ as an equivalent i 
form of the speech proficiency assessment, 30 nlnth-qrade students of 
varyJng abl Htles .were administered both tasks. Each of their performances 
wre evaluated by th^ee raters using the appropriate rating Instruments. 
The correlation between the average rating on the employment Interview 
~ and the average rating on the public hearing assessment was .697. 
.'This may be regarded as a s7jltably strong relationship for performance 
tests of thls l^Ind. 

equivalence between topics . In the public hearing assessment 
students are able to serlect their topic from among three agenda Items. 
*lt ts possible, however, that some bias may Inhere In the choice of 
topic. For example, some topics may be^lnherently easier than others. 
Or raters. may be more Impressed by some subject matter compared to some 
other. Such problems have Insinuated measurement error In large scale 
testing of writing ski I Is (Rosen, 1969). Tables 3 and 4 present ' - 
summary statistics broken down by topic for two admlrilstratlons of the 
public hearing assessment. Two types of analyses were performed to test 
the significance of choice of topic. Total scores were Subjected to 
one-way analyses of variance (ANOVAs) with unequal numbers of subjects 
tested In the -three levels of toplc^. 
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. . ^ YABLE 3 

Summory Statistics: Public Hearing Assessment Scores by Topic 
(Original lO-ltem Instrunent^ Videotape Rating MethodP 





Topic 1 
•<n«66) 


Topic 2 
<n»22)"' ■ 


Topic 3 
^ <n«t8) 


mean 

standard deviation 


. 22.92 


23.54 ' 


22. '36 


4. -94' 


3.56 


3.82 


mode 


15.00 


24.00 


20.00 


median . 


23.00 


23.00 


20.00 


achieving criterion (?) 


30.30 


31.80 


• 27.80 



Potential range: 0-40 

TABLE 4 

Summary Statistics: Public Hearing Assessmant Scores by Topic 

1 

(Revised lO-ltem Instrument, •'Live'' Rating Method) 





Topic 1 
(n=43) 


Topic 2 
(n=38) • 


Topic 3 
(n=l6) 


mean 


28.47 


27.63 


27.80 


standard deviation 


5e30 


5.23 


3.14 


mode 


32.00 


21.00 


27.00 


median 


29.00 


28.00 


28.00^ 


achieving criterion (JS) 


76.74 


71.05 


86.67 



Wtentlal range: 0-40 
• • • 

The rer»utts of these ANOVAs appear In Tables 5 ?jnd 6. While topic I 
(open campus for lunch) was ^the most popular, differences among top! 

* ■ » * 
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TABLE 5 

AMOVA of PubllclHoarlng. Assessment Scores by Topic 
(Original 1 0-1 tern Scale, Videotape Rating Method) 

Source of Variation df, Sutn of Squares rtean Square 

Topic 2 14.113 7.057 .346 

S(Tbplc) 105 2100.142 20.390 

' TABLE 6 

ANOVA of Pub.Uc Hearing Assessment Scores by Topic 
(Revised lO-ltem Scale', "Live" Rating Mathod) 

Sdureo'of Variation rif Sum of Squares Mean Squ?re F 

Topic 2 16.991 8.496 .349 

S (Topic) 93 2266,247 24.368 

means were not statistically significant at the .05 level. 

As a second attempt to verify that choice of topic Is not a po- - 

\ 

tent factor In this examination, students were cross tabulated accord- 
ing to topic (at 3 levels) and mastery (at 2 le>)e|s. I.e., achieved 
the criterion score vs. failed to achieve the crlteryon score). This 
score tabulation was conducted for the two test administrations men- 
tlonedftln the proceeding paragraph. (See tables 3 and 4 for frequencies 
of mastery broken down by topic.) Values of the Chl^ statistic were 
calculated l"n order to tost for statistical ly significant relation- . 
, ships between mastery and topic. choice. For the videotape method eat- 
ing Chl^«.078 (2 df, p>.IO). For the "live" rating Ch|2=|,45 (2 df, 
p>.IO). In neither trial was a statistically significant relationship 
ilwVealed. 



effects of snoaktnn ordnr . In the public hearlnn assessment, 
speakers are ^Iso audience members. It Is possible that later sneakers 
right be aided because they have an opportunity to Include material 
developedTby-earl ler speakers. Indeed^ observation confirms consider- 
able repetition of earlier arguments. Also, speaking order dictates 
the order In which judges rate the speeches. It Is possible that 
some fatigue factor^lght systematically affect the raters' performance. 
The correlation between, speaking order and total score was calculated 
In order to revejJ these potential effects. For the videotape method 
administration to 106 ninth-graders, the correlation coefficient was 
.1325 (p .05). For the "live" rating of 97 ninth-graders, the corre- 
lation coefficient was .3119 (p .01). A statistically significant 
order effect emeroed for the "live" rating, thouqh In neither case 
did speaking order account for more than 10? of the variance In total 
scores* It Is likely that the statistically significant order effect 

0 

found for 'Mlve" rating Is attributable to rater fatigue. Were the 
order effect due to students learning from preceding speakers. It 
would have been more marked In the videotape method rating as well as 
the 'Nlve" rating. 

Inter-rater reliability . As discussed In the previous section 
on training raters. It Is necessary to demonstrate that Independent 
Judges can reac^ similar decisions in their use of the rating Instru- 
ment. Two procedures tested agreement among .raters. oFlrst, total 
score correlations between pairs of raters assigned to the same stu- 
dent were calculated. The averane Inter-rater correlation derived 
from the videotape method ratlhg of 106 nf nth-graders was .8246 
(p<.OOI). 'The average correlation between raters derived from the 
'Mfve" ratlncj of 97 n I nth-graders was .7239 (pcOOl), 

4Q , • 
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In tho spl rlt- of crfterlon referenced testing, a second analysis 
examined the decree to which raters acireed In their classification of 
students as masters vs, nonmasters. Student scores were cross- 
tabulated for mastery (at two^ levels, l.e,„ fhose who had attained the 
criterion score vs. those who failed to attain the criterion score) 
and rater (at two levels). The resulting 2^.x^2""contln'gency-%abte was 
subjected to the Chi procedure to ascertain If the relationship between 
rater and mastery decision was statistically significant. The contin- 
gency table for the videotape method rating of 106 ninth-graders appears 

in Table 7, while that for the *Mlve*' rating of 97 ninth-graders 

2 

appears In Table 8. The obtained Chi for the former analysts was 46.463 

*♦ • 

TABLE 7 

Cross-Tabulation of Rater by Mastery Dec! s Ion 
(Original IO-lte>n Scale, Videotape Rating Method) 



Rater 



Rater 2 



mast^y 



nonmastery 



mastery 


nonmastery 


27 


10 


<25.5?) 




6 


63 


(5.1%) 


(59. 4?) . - ' 



(p<.OOI) demonstrating strong agreement between raters. For this adminis- 
tration, 15 of the students were cross-classified by raters, passed 
by one rater but not by the other. This percentage of cross-class I fl- 
catlons represents also the frequency with which a third "floating" 
rater was needed to resolve discrepancies. For the "live" rating, Chi = 
43*893 (pC.OOl) analn Indicating a strong relationship among mastery 
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Cross-Tabu I afl oh: of Rater by Mastery Decision 
(Revised tO-t tern .Scale, "Live" Rating Method) 

Rater I 



Rater 2 



mastery ' 



nonmastery 



67 
(69. t!?) 


4 

(4.1?) 


3 

(8.2!«) 


18 

(!8,6?).' 



, decisions. Raters cross-classified 12. 3? of the students using the 
.."live" method. 

. . * / 

Data were aFso collected regarding Inter-rater rel lab! I Ity of the 
job Interview ratings. Three raters evaluated each of thirty nlnth- 
graders participating In this assessment task. Simple correlations be- 
tween the scores assigned by these three raters were .506, .605, and 
.956. The reliability (Cronbach^s alpha) of the average of three ra"^- 
Ings was .869. While the lowest of the Inter-rjjter correlations was 
unsatisfactory, the highest of the correlations provides reason to 
believe that adequate agreement can be reached In the use of the employ- 
ment Interview rating Instrument If more Intensive training Is pro- 
vided. 

cultural group bias. No data were collected concerning the cul- 
tural or ethnic Identities of students in this project. Therefore It 
was not possible to ascertain If the assessment procedure was biased 
against any particular cultural group. Any future extension of this 
project will need to examine .this Issue. In doing so, however. It 
^should be noted that bias Is not empirically determined simply by not- 
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Ing If ono particular i^roup scored more poorly than another. Rather, 
a test Is biased If test scores over- or under-predlct scores on some 
criterion refei::pncmg me^asurlng (e.g., classrQom ratings of typical 
communication behaviors), and If these erroneous predictions are ob- 
talned for one. cultural group, but not another • Furthermore, cultural 
group Identity must not be confounded with ability level grouping In 
any such examination of bias. Finally, It may be noted that the 
communication task and evaluation criteria were constructed with an 
eye toward avoiding culture-bound standards, other than those which 
may be Inherent- In any institutionalized test of communication skills 
(cf, Rubin-, 1980). 

WHAt RESOURCES ARE NEEDED TO CONDUCT THE ORAL COMMUN ICATONS ASSESSMENT? 

As In the case with any type of assessment activity, one of jthe 
major considerations in oral communications assessment is the time 
element. Since the oral communications presentations are r^ted by 
teacher^, even more questions must be addressed concerning time. When, 
for example, are the teachers to be trained as raters? When will thd- 
•teachers rate the oral presentations? 

Rater training can be provided dviring the school day and In the 
evenings after school. . If a consultant has to travel a great dis- 
tance, however. It Is often difficult for the consultant to spend 
hIs/heIr time traveling for a two or three hour v/orkshop after school. 
One approach 'Is to work with one group of raters during the first half 
of the school day and work with a different group after lunch. Using 
this approach, one substitute can cover for two teacher^-. 

If the rating Instrument has-been "de-bugged*'^rior to calling 
the tgachers.ln for rater training, approximately 'four hours may bo 
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sufficient* As stated' in an earlier section, the selection of teachers 
as raters should not be llrjilted to the English department. ^ Also, as' 
Indicated earlier, some time must be Initially devoted to a '*we believe" 
discussion about the purpose and nature of oral communications and oral 
communications assessment* Video tapes of students of various ability 
levels giving oral presentations are needed for learning how to use the 
rating Instrument. Inter-rator reliabilities should also be computed 
during the training. 

If the "live" method of rating (as opposed to video taping the 
student presentations) Is to be utl I Ized, ^the raters should have some 
practice aslng the Instrument In a classroom before having the pressures 
of rating large numbers of student presentations; 

' Obviously the time needed to rate a population of students will 
depend on the size of that population. Generally, a class of 25 stu- 
dents can be rated during a 55 minute class period, however. Due to 
the fatigue factor; teachers probably should not devote more than three 
periods a day rr^tlng students* oral presentations. 

For .the sake of convenience, a class such as English In which most 
students are enrolled can be used for the rating sessions. The teacher 
of that class does not rate his/her class even If he/she Is a trained 
rater.^ Unless the presentations are video-taped the only equipment 
needed Is a speaker *s stand and a audio-tape recorder to record the 
presentations In case a third rater Is. needed. 

Raters In the Glynn County School System decided students were not 
as relaxed while telng video-taped. In addition, the number of video 
cameras and recorders available limited the number of presentations that 
could bjB^gtven simultaneously. Therefore, the decision was made to 
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r?ite the presentations while they were being given. 

. Aripther^resource needed; when preparing for the oral communications 

^ . ' ' 

assessment process is consultation services. An expert In both .oral 

coiTfnu^^^^ oral communications assessment, plus a knowledge 

df ^the CBE movement, will be needed to help identify the rating Instru- 

fiient, train the raters, organize the rating sessions, and assist in 

analyzing the results. tKis consultant should also assist In identify- 

log gaps In the oral communications strand In the curriculum and with 

Identifying strategies for teaching oral communications. 

' The monetary cost for preparing and conducting oral' communications 

assessment is relatively small. Substitute teacher pay to provide for 

release time for raters, stipends for after school workshops, .and 

honorariums and travel expenses for technical assistance are the major 

expenses. Some of these expenses would be incurred only during the 

first developmental year. One approach to estimating the cost of such 

a project would be Iro: — — — ^ 

' I. Decide how many raters are to be trained. Keep In mind 
you will need a team of two teachers to rate about 75 
students a day. How many days do you want to extend 
-the rating process? ' 

2. How will the classes of the raters be covered while 
these teachers are rating the presentations? If 
substitutes wlTl be used, allow one substitute for 
every two teachers (one In the morning and one In 
the afternoon) and muJtlply the dally pay by the 
number of days needed to complete the ratings. 

3. Add in $1,000.00 for 5 consultant days^' This will 
vary according to how much time Is. spent In Identi- 
fying, devejoplng, or modifying a rating Instrument. 

4. Decide when the raters wl.l I be trained. If It Is 
during the school day, add substitute pay. If any. 
If this will be done after school;^ add In hourly stl- 

.pends of approximately five-dollar an hour, 
• * , ' * - 

5. List the cost of other nilscel laneous items such as 



audt^o t^pes, (can: be jjseci, fwr^^ :video tapes 

fbr .ana vl deb 

^'t^^T^Mi" ^^^^ ! ta!^%^^pH^^ ahd repro* 
dijc^ for tHe* :ratt nstrOr^^ 



^PRQGRAMr r ; ^.^^..^ . .-v^.^, > .^^^ 

* . %yera the resbrvat ions f§l^^^^ teachers and students at the 

'otrfset iof the project were relieved after/students participated In the 
assessment activities. In the early training sessions, for example, 
Sofne t^ea^^ the belief that Vmany of their students would 

not^;be-iabie t^ that.^' ^Ince f^^^^ If any,. 

. previous exper a groupf teachers were somewhat 

skeptjcal; . . . 

Although a small number of students refused, most students did 
•participate. Some teachers were surprised that their students did very 
wel-i* Some students Indicated at the end of the school year that this 
experience had .been more helpful to them-thah_anythlng else they had 
done at school that year. Some senior English teachers give their 
students an opportunity to try the publ Ic hearing simulation without 
the benefit of trained raters, and the students thought Itwas fun.. 
Having, less pressure to "perform no doubt made some difference. As in 
the case when many changes are made, seeing can make believers of 
doiibters. 

the oral, communication assessment program had a generally positive 
effect on overall staff development, the experience of having teachers 
from the English, social studies, and career education departments no 
doubt had a posW'tve effect. If prompted a more harmonious working re- 



fStlpnshjp between departfnents. Rating teams were often composed of 
tMcherS from different departments both during the training, and during 



;actua,r Tdtlng. Moreover, Instrument development and rater training • ^ 
-sesSrtp^^^^ had the^qual l^y of :f nrsorvlce. edticaf Ion for teachers who 
^prciylpusly had no exposure to principles and rftethods of speech communl- 
cation; ^ 

.During the devdlopmDnt_ of the rating scale and training sessions 
the thought was ofton expressed that by developing assessment procedures 
before curriculum development and Instructional activities we were getting 
"i-he.cart before the horse," St-nce It takes monihs to develop such an 
assessment procedure howeyer, and because assessment procedures 'for the 
other;! I fe role aVeas were In the final stage of development, seff-Impose.d 
-pressure was felt, ■ ^ . . 

Developing an assessment Instrument does obviously force the Identi- 
fication of what It Is that the student must be able to do. And this 
identification of objectives is a first step in thelnstructlonal pro- 
cess. Teachers who have been fnvolved. In. the development of the rat- 
ing Instrument and the. actual rating of oral communication performance 
wjJJ also be made responsive ^o making the necessary changes In their 
classroom practices. Having this type of Involvement will tertalnly 
motivate >he teachers to provide the necessary Instruction, 

Recent curriculum revisions have been ^ade to Include Instruction 
In speech In the English courses and revisions will be made In govern- 
inent courses during the 1981-82 school vear. Staff development activities 
wtllalso Include Instructional strategies In oral communications,' 

the greatest Impact of this project was to adopt' the procedures des- 
cribed In Jhls'^paper as the means to assess oral communlcatlonr compe- 
tence. During the 1980-81 scho6l year aM ninth and tenth grade students 
were given the opportunity to be raHd, Since these are the first classes 



that fwjst shpwjcor^^ In ^^M the areas Identified as-mlnlmum, this 
assessirient was "for reaU* A vast-^majorlty In both of these clasises 
passed the assessment* 

Remediation wHI be handled as a regular part of the English 
class since oral communication Is Included In all courses. Even those 
who passed will be able to profit from additional' Instruction. Those 
who fall to pass a second time will be given more Intensive remedia- 
tion which will occur by pulling students from class and after the 
.regular school day. - * ^ 

A report of the results were sent. home to parents, • See Appendix 
M Tasks (Skills) In which the student scored lower than 3 werelnarked 
as needtrw^mprovement. This report wl 1 1 also faclHttate remediation. 



is 



43 



In .brief, this pilot project demonstrates that larcje scale cri- 
terion pefer«nced testln?] of h-lTlhjschoo.l students' speech proficiency' 
15 feasible! The cornmur Icatlon' task, a simulated public henrln.q, corr*' 
fomied to the specifications of the district for^sampllnn extended 
discourse In a formal," "life role" situation. It provldos students 
with a sense of purpqse and context, and permits* exercise of a full/ 
range of communication skills from Invention of subject matter to vocal 
and gesturrJI deHvery:/ At the same time, ft provides the structure 
necessary to control extraneous sources of variation.- Moreover, the 
,^task proved meaningful and manageable to the majority of students. 

The rating Instrument devised for use In conjunction with this 
assessment task was a type of rhetorical trait measure. It provides 
detailed Information about students' performance which can Inform fjlaco- 
ment decisions for subsequent remediation, as well as competency 
certification and pronram evaluation decisions. Jt exhibits con- 
tent validity within a limited content domain, and empirical J nvest I - 
' gat I on revealed strong Internal consfstency. Criterion referenced 

9 

validity was also demonstrated. Given sufficient train Inn In Its use, 
raters were able to attain substantial agreement In' assigning scores. 
It should b,e noted",- however, that the videotape rating method was 
somewhat superior to "live" rating witb respect, to reliability. 
Particularly troublesome Is the apparent rater fatigue factor In "live"" 

ratings. *- ' . • 

This pilot project also developed an alternate form of the speech 
proficiency assessment, the-employment Interview 'task. This proce- 
dure 'displayed adequate equivalence with the primary assessment task. 
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the ^public hear1n<} situation* The rating Instrument associated with 
this alternate fom possessed strong Internal consistency* Results 
suggest thBt with more substantial training .than that proyt fled In 
this project. Judges can attain acceptable^ levels of Inter-rater reli- 
ability. - V ' 
• The development and field testing of this assessment procedure 
^ appears to have exerted salubrious effects on oral communlcatlr)n 
v^r^ Instruction In local high schools. At very least, teacher .and student 
participants have become sensitized to some fundamental oral commuhl- 
cation skills. Based on Informally elicited remarks, the teaching staff 
seems to have shifted Its attitudes toward oraJ communication education 
In a positive direction,^ Most significantly, the speech assessment 
program has Initiated a procesfr^ of curricula*" Innovation whereby tea- 
chers are providing more deliberate Instruction In oral communication 
than was previously tht> case» 

The costs of measuring and certifying s+udents' speech prof Id ency 
In this manner are substantial. The procedure d'^mands considerable 
allocation ^)f student and staff time, the latter entailing allocation 
of funds for substitute teachers-* Certainly an Indirect, multiple 

Y 

choice test of communlcatlbn skills would be less costly. This option, 
however, was never pursued by the district. Previous experience de- 
monstrates that such tests are difficult to construct and would, If 
Implemented, likely have deleterious effects on- Instruct Ion ^ubln, 
1981.). In balance, the .costs of administering an examination pf oral 
' communication skill are outwelgh-?3 by tlie positive outcomes. 



FURTHER CONSIDERATIONS 

alternative evaluation methods to control costs. 
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The assessmont pnocedure described In this report Is the product of 
iir»ny cq^ . Ideally,, for exariple, student performances would be 

^dmpjecJ. In p variety of communt cation situations rather than In one 
primal^ and one, altematlvev-task/ Similarly adoption of "live" rating 
Instead of the vlfieofape methcd* sacrifices a degree of reliability, 
but maintains nevertheless an acceptable level of psychometric ade- 
quacy. Some additional savtng^of ttno and expense could bo realized 
by utilizing a general Impression rating scale. General Impression - 
scales, frequently employed In evaluating written composition (e.g., 
Hudson i Veal, 1981), require that judges render only a single over- 
all rating o4 a performance, frequently along a ftve-polnt scale. 
This rating method speeds up the scoring process, and may even epchance 
Inter-rater peilablltty. What Is lost^ hoy/ever. Is the detailed in- 
formation that can guide, subsequent student placement and currlcular^ 
adjustments. In, any event, use-of genera! impression marking caVinot* 
.reduce rating time beyond the length of student speeches. 

( 

Two other alternative evaluation procedures shift the burden of 
evaluation from a cadre of raters to students^ classroom teachers. 

One such method, employed In Massachusetts public schools, requires 

>> 

teachers to rate students' typical communication skills as observed over 
time In' the course of classroom Interaction (Massachusetts Department 
of Education, 1979). Those students who fall to pass this Initial 
.screening by classroom teachers are then subjected to a series of 
specific assessment, tasks administered and rated by a trained evaluator. 

• A second method which shifts the burden of speech assessment to • 
classroom teachers demands* In-class performance tests on. specif led. 
communication tasks. Classroom teacher?; act as test administrators 
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and raters i 1 1 lustratlve of this administrative arrangement are a 
procedure developed by the Pennsylvania Departrnent of Education (1079) 
Which Includes a single narrative task, and an evaluation program man- 
dated (by the/VermohJ" DeparffTont Education (?977) In which class- 
room teachers administer and evaluate a series of communication tasks. 

These methods, which call upon classroom teachers Instead of 
selected raters. Incur a great risk of ynrel labl I Ity. - Even assuming 
TOSslve In-servlce training programs, classroom teachers cannot shed 
tte If expectations and preferences, factors which must Inevitably 
color their judgements. "Without the check of a second rater, rell — 
ablllfy Is endangered beyond acceptable limits. Moreover, It Is not 
clear that such arrangements constjtute true savings, for teachers 
already, pressed for time must give up some Instructional efforts to 
accommodate to^ these assessment related roles. 

assessing I Istenlnq ski I Is^ The goal of this project was limited 
to develop.Ing an^ field testing a procedure for measuring proficiency 
In speech. The project has not considered the complementary and equally 
Important goal of assessing listening competent? I es. In all but the 
iTOSt formal discourse, of course. Individuals shift ^lamlcal ly between 
the roles of listener and speaker during the course of Interaction. 
Such Is the case In'the employment Interview task. In fact. We spend 
riore time llsten^lng than In any other waking hour activity. A complete 
program of oral communication Instruction cannot neglect listening 
skills. 

Assessing listening skll|s Is a less complicated and costly 
enterprise than testing speech proficiency. Tests of listening per- 
formance can be group administered and mechanically scored.' It Is 
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initial test devolopment which constitutes 25n impediment to mriasurlnn 
listening ability. For the most commercially aval lable listen-. 

Ing testjs measure only limited types of listening (Rubin, Daly, Dickson, 
McCroskey i ftead, 1981 )• f^any Qommerlcal instruments are merely 
reading or vocabulary tests presented orally. Greater validity inhere 
in tests which include listeninq for point of view, listening for de- 
cision making, . I Istening for instructions. Msteninn for inference 
making, and so on. Such listening examinations have been noncommer- 
c.ial..ly developed, must notably by Alberta^s Ministry's Advisory 
Cbmmittee on Student Achievement (1979), Massachusetts Department of Edu 

cation (I97<^), and Michigan Department of Education (1979), Explolt- 

r 

Inn these models. It would be quite feasible for a local jurisdiction 
to create and field test a suitable listening assessment instrur^nt. 

K'-tZ currlcular support. Enmeshed in the Retails of test adminis- 
tration and technical analysis, it Is -cat I too easy for a project of 
this kind to lose sight of Its ultimate purpose: to enhance students* 
communication skills. This report has notejJ that testing programs of- 
ten provide Impetus for currlcQlar Innovatlqn,^ It'would be unethical 
to pursue this, or any other, achievement tes+ ing program If such c'urri- 
cular support .were not forthcoming. 

Two f;;;neral points ought to be raised with regard to appropriate 
implementation of an oral communication curriculum. First, deliberate 
oral communication Instruction should appear at all levels of a student' 
public school career. The focus of the present project was on a htgh^ 
s&hool gr^uatlon competency test/ However, a sound foundation fpr 
tfiiBse exit-level skills needs to ^e ^et,eanly In the primary grades, ^ 
and developed throughout the intermadlate grades. It Is an error, for 
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example,. to presume that persuasive discourse Is a f It/subject only 
for secondary level Instruction, Younger children also possess per- 
suaslve skills which, might be refined and enhanced (Pich^, Rubin & 
Mlchlln, 1978) • It may not be wide to assign formal public speaking 
assignments to second-graders, but second-graders can nevertheless role- 
play persuasive situations with which they are familiar, Slmllarfy, ^ 
younger children can learn thjB* value of orienting listeners to the pur^ 
pose of a talk, though they need not be exposed to the notion of a 
formal Introduction. , 

A second worthy point about currlcular support. concerns the range 
of skills to be taught* The public hearing and employment Interview 
assessment tasks are sampled from a broad domain of communication situa- 
tions, and they represent on^y a limited selection of communication 
co.:ipetencles. A well motivated oral commun^atlon curriculum would not 
merely prepare students to pass this competency <*^.3ml nation. Rather, 
comprehensive communication Instruction would encompass the spectrum 
of Interaction types from highly reciprocal conversation to highly 
^formal and ritualized speeches of appreciation. It would span a range 
of listeners from familiar peers to remote audiences composed of "the 
general ll^^d other." A superior communication curriculum would pro- ' 
vide students with, experience in a variety of communication acts Includ- 
Ing pantomlfJ», creative dramatics, small group problem solving, and 
pari lamentary discussion. It would Include units on nonverbal communl- 

' ■ * W 

cation, dialects and la^gua^e y3rlatref*>«ftropaganda techniques, and 
story telMng. In short, instruction In oral communication must recog- 
n I ze*i^a^ assessment procedures are^-lndlces of student achievement; they 
cannot- be 'a I lowed to^ circumscribe outcomes, of . learning. 
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. APPEND rCES 
APf?ENblX A 



;S6fne gti I d 1 nq. p r 1 nc t p I es 

U-l ;A commltrnent to assess oral communication ski Ms* is an *acknow- 
. ledgafrient of, the primary fmportahce of speaking, and listening 
• Tn'^our lives, . Oral cbmmuhlcatron is a tool for Influencing arid 
.icooperatinr} wjth others, for discovering our world, and for self-- 
ej<pressloh. It \s also the foundatfon upon which literacy Is 
"built. ' , • . 

1:2 A commitment to^ assess oral communication skf.lls entails a 

'commitment to incorporate deliberate Instruction In speak5nq and 
listening throughout the grades. It ts true that we learn to 
communicate throuqh everyday experience. .But effective and flex- 
ible speech communication skills are cultivated in teacher- 1 earner 
Interaction. 

1.3 Oral communication takes p\ace within a social context. Artifi- 
ciality and an exclusively evaluative climate inhibit speech. 
Assessment procedures ought to involve t35j<s which ask students to 
communicate for real purposes in as natural.i'stic a setting as - 
possible, 

1.4 Criteria for evaluating speech communication are functional. They 
are defined in terms of effectiveness and appropriateness within 
given interactive situations'. Absolute standards or standards 
which do not refer to the unique quality of each qommunicatfon 

. setting, performance^ are rarely applicable. Nor should stand- 
ards of written lanquage be directly transposed to oral assessment. 

L5 Evaluators of students' oral skills must be familiar with the 
fundamentals of speech communication. They must be trained to 
recognize the demands of the assessment tasks and to rate students 
In a consistent manner. 

1.6 Assessment tasks must be designed to provide students with feed- 
back concerning their strengths and deficiencies. Feedback should 
be phrased In a supportive manner and serve as the basis for 
additional developmental instruction. 

Overview of assessment^tasks 

Tasks Involve a number of life roles identified as of primary importance. 
They are 'set In a variety of Interactive contexts (one-to-one, one-to- 
qroup, one-to-many). The tasks Include demands for both speaking and lis- 
tening. They Incorporate several communication functions (inforrmlng, 
questioning, persuading, establishing relationships). The District may 
select ai I or only some of the tasks. The tasks may be modified depending 
on the resources available. 
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:3.; ^ yfelSphon jhfdi^f Ilon r :" T^^^ . y 

,3. i; Procedure*. Student makes^^^ to the State^ 

^ Labdr jlfepart^ to Inquljre^a^ Sumnier employrhent oppor^ 

tuWtle^^ Sfud^^^ listens for and » 

responds to re^^^ fop. cja.r^^ request addltlpngl infpr- 

'inaf (on on pros^^^^ listens for Instructions., pro- 

per I yYconcJutJes^t^^ ' . ' 

3.2 Evaluation criteria. Performance indicators are rated on 4 point * 
, scafQ. (5ee section '9) • Criteria Include, proper Iritroductlpni 
specification of Interests and qual if icatlons^ response to request 
for ,cf.arl;f Icatrlpn., conveys pwn need for additional jnforrhatloriv 
, ypice rate, volume, Inflectlph, appropriate J IngulstIc styJe, and 
cpurteisy. . 

4, \il6t> /Interview: Task B . ' ^ 

4 J, Procedure, Student Interviews for choice of. poslt^lons'at a fast 
food restaurant. Procedure Includes filling out application form. 
Questioning - provides opportunity for se! f-dlsclosure and self- 
ana fys IS' and for takinq the role of another Individual. See 
attached sample Interview. 

4,2 Evaluation criteria. Performance- Indicators are-rated on 4-point 
scale; (See section 9). Criteria^ Include appropriate response 
to narrow questions, elaboration In response to open questions, 
creating opportunities to show Interest and Initiative by ques^ 
tloning the Interviewer, tactful Inquiry concernl'ng salary, start- 
ing date, voice rate. Inflection, volume, appropriate eye-contact 
and nonverbal gestures, appropriate' I Ingulstic style, and proper 
conclusion, ■ 

5, Giving Instructions : Task C. 

5.1 Procedure, Student learns a vocational technique by viewing a 
film or participating In an auto-lnstructlonal module. ' Examples 
of such technique Include (a) preparing mall for bulk malting, 
(b) rudimentary operation of keypunch, (c) assembly of simple 
.mechanism (e,g, wind-up clock). Student has opportunity to prac- 
tice and demonstrate mastery of technique. Student then teaches" 
this technique to a peer, 

5.2 Evaluation criteria. Performance Indicators are rated on 4-pOint 
scale, (See section 9), Criteria Include, student gives orienta- 
tion or overview of task, student gives Instructions In proper ^ 
sequence and In appropriate units of Information, uses clear langu- 
age, asks for and responds' to feedback, student explains purpose 
for each component subtask, student does not rely on nonverbal • I nd I 
cators, speaks with prope/* rate, volume. Inflection. / 
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-6.4 Rrbc^dur^^ Students .paH'lc^ In JeaderJ ess peer nroup dis- 

cusslort ; in^^:o dVleTnmas or question, of poJ Icy ^. 

Wld^h I h- thai r ranj?e ^of experl ence and; ] nterest ; . Examp | es of such„ ^ 
^tbplcs;- :Inc^ "What :shpujy you; do If you discover a friend ex- 
perlm^tln^ "What can high school students dp 

. fbr-them^Ives. to market of the coming decacje?," 

'HtoW s>i6uld we ch^ lifestyles to adapt to the need to con- 

serv^ energy?'^ Students must adopt roles for effective group 
functioning,, clarify th^ develop potential solutions, evalu- 

ate solutions. . ] ' 

6.2 Evaluation criteria. Performance indicators are rated for each 

-group paHrJclpant'oni a 4-poInt scale; (See section ?). Criteria 
Incfude, does not dorn|hat^^^ seeks' others' opinion^, 

exRresi&es own qpl^ at appropriate .p6int, paraphrases others' ^ 
(XJntBlbutJbns, expresses disagreement and agreement > clarifies 
topic. Identifies points of contention , and. cprnmonailty, gives log-, 
leal support^for own point of. view by means ^f 'example and other 
forms of evidence, expresses evaluation of others' ideas construc- 
tively, voice rate, volume. Inflection, nonverbal gestures. 

7. Public hearing : Task E » - ^ 

7.1 Procedure. Students parti'cipate in a simulated pubUc hearing 
such as^mioht. be conducted by county Gommiss loners. Proposals 
under consideration include construction -of an industrial air- 
port, abolition of all zoning restrictions, increased property 
taxes for Improvements in fire department and county hospital. 
Student selects one issue on which to testify before the hearing 
officers. Student decides his/her own^poinJ* of view. Time limit ' 
on statement Is 3 minutes. 

7.2 Evaluation criteria. Performance indi-cators are rated on a 4-point 
scale.' (See section 9). Criteria include, expresses Indentif iable 
point of view, gives reasons, supports point of view with Infor- 
mation, arguments are adapted to the point of view of local govern- 
ment administrators, llnqulstlc style' Is appropriate, states poiht 
of view clearly In introductory comments, concludes appropriately, 
uses appropriate eye contact, nonverbal gestures, voice rate. In- 
flection, and volume. 

8* Critical listening ; Task F • • . 

8J Procedure. May be conducted Individually or in group. Students 

vTew a videotaped condensation of a typical newscast. 'Tape Includes 
segment of political speech, biased report of news Item (e.g. high- 
way construction project), advertisements, sports report blending 
into editorial comment, weather report. In group or Individually 
.students are interviewed to determine their discrimination between 
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* :fac^;3nd pplhlon^i re^^ Inferences from Infojr- 

mat loriiglyeHii ^a^^ 

8i2 €vdluatf on criteria. Performance Indicators are rated on a 4-polnt 
acale. CSee section 9^^^^^^^^ ^Grltfenla liiclud 
* opinion ♦ Idenflf les po^ view and Infers motivation when 

apprp'p^ correctly catenorlze^s persuasive appeals (e.g.;, band- 
^- • Wgoh approa^^^^ gl Iftering ^gerieraH^^^ Incomplete statistics), ' ^ 
Infers proper personal action (e.g., on basis of weather report), 
' Independent of -moral judgments. , . * 

9. Evaluation procedures 

l^^^^x\^^ r^^ers record their judgment's on Instruments designed specif leal Ly 
-f6rje,ac^ Each performance Indicator Is rated on a four-point scale. 

• as .follows: ; ■ 

• • 

1 - Remember to do this next time. See your teacher for further 

suggestions for Improvefi^nts, 

2 - You have this skIM , but you can 5*0 this more effectively or 

consistently next time, ^ - - 

. 3 - You did well, but In order to Improve your communtcatlon skills, 
polish up your use of this ability to be extremely effective, 
4 - You^ve mastered this ski 11,^ If you can remember to use this 
ability in other situations, you wTl I be an effective communi- 
cator. 

A rating of 3 or 4 Indicates that the student has exceeded the performance stand 
ard,. Whenever possible, students will be audlo-or videotaped so that they may 
review their performances In conjunction with the feedback/evaluation forms. 
Preferably two raters will evaluate each perfonnance. When ravers disagree, . 
the taped performance can be used by a third evaluator or team of evaluators 
for a 'final determlnatJon,^ Evaluation procedures a I low students who fall to be. 
retested .at frequent Intervals, recognizing that oral communication performance 
Is affected by many,>^Hables and may be Inconsistent from session to session. 
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y**"* per.on.Xly offeniiyiv 

tCMON^ 'IMfORMAXIOli . 



P.te 



Sbei.! Seeuritv yuMber 



First 



Middle' 



Age 



Sex 



Street 



City 



Stete 



Street 



■Father*. Occupation;, 



City State 

gather* a Education; 



Zip 
"zip 



Mother? a Occupation; 
Phone Wuinber 



Mother's Education; 
Citizen of P.S.A. Yes Mo 



■Pate of Birth 



JelshL 



Weight 



Color of 
HairJ 



Color of 
Eves 



EMPLOY)^ DESIRED 
Fbaitibn; h6at/ho9t ga« cashier 
Pava and Hours Availab le ; 



cook 



D«te You 

Can Start_ 



Siil.ry 

De'aired 



■Are You Eaploved Wow? 



If So' Kay We Inquire 
of Your Prgscnt "Employer 





MasM and Location -of S^ool 


Ye.ra-' 

Attended 


Date 

Graduated 


EDUCAftOlf 


GrasMDar School 






f 




High School 











£j^(;^v|cf Athletic* Fraternal, etc.). 



~ Iradti -Bualneas 
{ 'or Cprrtspondence 



and Location of School 



FORMER EMPLOYERS (List below L«.t Two Empluy.rs, Starting With Laat One Flrat. If 

No Previous Employer. Write IWNE^ 



Date - . 
Month md ^«<or 



■ Fron 



To 
'Tron 



To 



REFERENCES 



Haac and Address 

of Employer 



Salary 



Position 



Reason for 
Leaving 



: Give Below the Ma»s of T^o Persons Not Related Ti You. Whoa You Have 



Known At Least One Year. 



N 



1. 



2. 



Town 



Business 



PHYSICAL RECORD; 

List Any Physical Defects 

Were' You Ever Injured? 



Have Tou Anv Defects In Hearing? 



In Vision? 



SIGNATURE: 




00 NOT WRITE BELOW THIS LINE 



Appendix B (continued) 



AMVcr this question coinpletely. 



-. APRENOW C 

' • , - ' . Script for iob IrP^rvIew Task ~" 

. Thartk you 'for fllUng'out all these^ forms* NowJlet^s talk a little bit 

jubbut ,you. Have you ever Interviewed for a job before? Well you know J"hat 
Jl. thjB most Important think for you to do Is to relax and^act as natural as you 
-can. 

^ \ ^ • . * ✓ . 

\\. I see you prefer to work as a 

a* Cashier. Any particular reason? - - . ^ 

How well do you do In math? ^ 

• Have you ever worked with money before? 

i *^ - How do you feel about bel no- responsible for large amounts of money? 

/ * <? . 

Our policy Is that cashiers are held responsible for any shortaqes. 



That means we ask for a reimbursement If you come up short. 

i ^ r ■ ^ . ' ■ 

How do you feel about that?^ 

How are you at talking to people? - • 

« « 

b* Cook* Any oartlcular reason?" ^ 

Have you ever been a cook before?^ , / ' . - 
What were your duties/What do ypu know about the duties of a cook? 
How'd you like It/How do you think yoii'J like^ th^t job? 

Cooks have to be able to keep a lot of order's, i often special orders, 

i ' 

In their heads on once. How do you feel about that? 
c* Host/HosteSs* Any particular reason? . ^ ) 

Have you ever flone this job.beforef , 
"Can you describe your duties/What do you know about what a host/hostess 
does? ' ' - 

What did you like about the Job/Why do'you think you'd like this job? 
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.Mi. a. 



c. 



Appendix C (continued) - ^ 

:in >thl).> restaurant the. hosteSs/host hires and supervises al I waiters 
arid waitresses and makes up the work schedules. Can you tell me what 
qualifies you for thls sort of responsibility? 

I Sep you're working somewhere else* How long have you worked there? 
How do you *eel about your job? 
TeH me about your duties. 

Do you think you can handle both/V/hy do you want to leave? 
How do you get along with the other' people on the job?. 
How do you get along with your boss? 
What would he tel I me about you If I asked him? 
b. I see you used to work at » 
How long did you work there? ^ 
What were your responslbl ! Itles on that job,? 
Did you net along pretfy we I I- with your co-workers? 
What about your boss? . " 
How would you feel about working there agaFn? 

If I were to qall your boss for a. recommendation, what would he tell 
. about you? 

Tell me ibout yourseTf as a student (for those who are hot working 
somewhere else), * 

» 

How do you I Ike ^school? 
You make pretty good grades? 
What are your favorite subjects? 
How do yot* net alonn with your teachers? 
What about your principal? 

Do you have any favorite ^^dults at your school? „ 
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. Ajipendtx C (continued) 

if I called your principal (or one of your teachers), what would they, 
tei I me about you? 

IV* Tell-*JTe a Htrle about these activities you listed hec6/! see you dldn*t 

like any activities here. What do you do frf your spare time? 
y. Tell mo some of the things you would like to do with the money you earn 

from this job, • . * 

VI. Md like you to tell me about an Incident that^vill show me what kind of 
. \ a worker you are. Tell me about a time when you really had to put out an 
effort to do something that was Important to you. You can tell about this 
Incident as If It were a story, with a beginning, a middle^, and an end. 
(PROBES - Use the following probes if student produces three 
sentences or less) 

a. Tell me more about why you needed or wanted to do that. 

b. Tell me-more about why It +ook such an effort. • 

^ c Gjve me a better Idea of exactly what you needed to dc^ 
d. How were ^ot her people acting while aM this was going on? 
Or* how dtd they react afterwards? 
Vll. Now are there "any questions you would like to ask me? 
VIII. Our Intervie-^ ]s finished now. thank you for your time. 



; . ; . APPENDIX 

Public Hearing 

* ^ ■ agInda , ' 

Ciis^for Cbunjty Bpaf d of ^Educatlbn * 



Tfe Custer County Sqardjpf fducatlon will -hear comments from students, tea- 
" *chers, parents, and other* concerned citizens* Those who wish to speak be- 
- fore the Board may choose- only one proposal. Lfmlt your comments ^o thr:ee* 

minutes. Take some time now, to think about what you wish to say to the Board 

-ftembers. * Prepare sdme notes or a sketchy outline to fake wlth^^v^^when you 

spfiak, ' ^ , • . . 

" Th9 Bpard will consider the following proposaJsV. - . . 

1 , Open- Campus for bunch ; 

The proposed* regal at ion would ^oermit students to leave the high schopi 
campuses during their lunch poniods.. - If ,the campuses were opened for ^ 
lunch, students would have nreater freedom of, choice ^bout their food. 
Howevef tardiness and, truancy might .Become greater problems. There are 
presently few .suttab'^ restaurants within walking^ distance of the schools-.. 

2. Eliminate Fund Raising ^In SchdoT ^ ' ^ 

Many students raise money for various clubs^and organizations by selling 
candy, cookies, and raffle tickets to their fellow students. This type 
of fund.\ralslng helps support such activities as band,' 4-H Club, and the 
^ .Debate Team. These sales are. so na^erous^that they sometimes Interfere 
wjth classes, .xind students^who are. asked to purchase the candy, cookies; or 

' raffle tickets may feel hara'ssoH. The proposed regulation would prohibit 

• . ' , , .. ^ . 

such 'sales' oh -school grounds. • . 
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Appendix D (continued) 

'3. . -Ban Students From Driving ^tq^.Schbol 

Under the proposed regulation, only students who can prove they need cars 
to get to their Jobs "after school would receive school parking permits. 
This would reduce traffic congestion In the parking lot and also'help con- 
serve gasoline. At the same tlme^ the cost of school bus service would 

^ • rise s i fghtly. ' ' « 
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APPENDIX E 
A STUDENT GUIDE TO EMPLOYMENT INTERVIEWS 



Nothing can substitute for proper "trajalng and skill In landing that 
Job"Of your dreams* But the way In which" you present yourself to. n pros- 
pective employer is another Important factor. Many a qualified person has 
nil.ssed out on an interesting job becaund he or she made an unfavorable im- • 
press ton. When an empl"oye relist bai'ween ^nvo^equa Hy ^ua4+f4e4-app-U-^ 
cants, the one who can communicate most effectively will have the winning 
edge. 



The application form-rs- +he--f h^st-^easkm-youJ^ave-tjCL^^ 
person you are. No-one wants to hire a person who writes today ^5 date in^^he 
space for "date of birth." An applicant Who does that shows carelessness, 
and could be a danqerous worker. Personal grooming may also be a cj-u^ as to 
the kind of worker you "will be.. We all make judgments about peopTe based 
on their manner of dress and physical appearance. 

" «, 

By the time you reach the employment interview, the errfployer already knows 
a good deal about you. For the employer, the interview is a chance to learn 
more about your personality and Interests, about how wtli you think "on your 
feet'\ and especially how you handle yourself wlj:hr^other oeople. No matter ^ 
how skilled you may be, you can not work ef fec;l:rvely unless you can communN 
cate effectively - to customers, co-workers^ and suppliers. At the same time 
as the employer is sizing you up, don^t forget to use the interview as an 
opportunity for you to size up the emplpyer. Is this a conpany in which you 
qan fit In comfortably? Will you fir>cr the job challenging? Of course you 
'ought not appear arrogant^ but an employer will be favorably Impressed by 
your Interest and maturity If yoji ask such questions. 

Here is a list of some af^he communication skills an employer may be * 
.looking for during the course of a job interview; 

SOCIAL RITUALS- Before the interview begins, do you greet the interviewer, 
state your name and .the purpose of the interview? After the interview, you 
thank and take your leave and state that you hope to hear from the Interviewer 
In the future? / 

RESPONSjyENESS- Do you answer all questions and even volMnteer relevent 
informatron on occasion?^ 

INFORt^ATIVEMESS- Are your answers relevent to the questions? Do your 
, answers avoid vagueness by supplying details, examples, and Information about 
1^ .V yo)^e\f In various situations? * ' 

INITIATIVE- Do you occasionally ask a question of the Interviewer or 
volunteer a comment? 

INTERPERSONAL t^ANNER- Are you self-confident. Interested, and appropri- 
ately respQctful? 
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Append I \ E (continued) 

l^NGUAGE STYLE- Can you speak fluently, meaningfully, and without too 
much slang?. ' - 

♦ ' 

'ORAL EXPRESSION- Do 'you speak ?n a conversational tone Of voice and U5e 
your voice to show your sincerity? 

SPEECH RATE AND VOLUME- Can you be heard easily? Do .ylou speak at a speed 
that can be followed without effort*? 

, ^ ; ' 

GESTURES- Do you use your' face, body; and hands in a natural fashion to 
supplement what you are saying? Do you look the interviewer ,{n the eye? - 

Improving your skills in these areas will help yo\j to becoma a bett^er 
communicator in all your encounters, including Job interviews. Probably the 
single most important piece of advice to carry with you into an employmen-b. 
interview is to be truthful and to be yourself. With that attitude; plus a * 
healthy dose of motivation, an interview can be a positive exf^erience* 
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APPENDIX F. 



A STUDENT GUIDE TO PARTICIPATING IN PUBLIC HEARINGS 



Our -goVernmGnt has grown complex and touches almost all areas of our 
lives. Cone are the days when a oorson could just set up a sign and go 
!ntov<buslnoss. - flost businesses must now conform to a number of qoverhmental 
licensing and safety regulations. Govprnijients now also provide services such 
as Social Security benefits and consumer awaren ess information that were un- 
heard-of .when your grandparents were young aduPfs; EducatliDTr-ts-corpulsory 
for all. You must take a specified course of study and fulfill other require- 
ments' In order to receive a certificate of graduation. 

^WsfW&y^\\'t€f^^^^'^OMBr\^s~\vr^\\^ f ace-of-b^l^-governmeR-t, _ Sorae=^± 

times they complain. But ours is a government of the people, for the people, 
and by the "peopled Often li is pur own fault If our government is not serv- 
-ingMUS-th4~way-weLfeel: It should,. . By. exercising our right to vote we can 
have an. influence on governmental decisions. Often times, various governrffint 
agenci^ wTTl "conduct public hearings so that citizens can express their vJews 
on various issues. You may be surprised to learn how much influence you can 
exert by .speaking up at such public meetings. 

The affairs of- local county government may not attract the attention we 
usually give to national politics, but in many ways local government has a 
more direct effect on our daily lives. County governments, usually headed by 
a E3oard of Commissioners, decide the amount of property tax you will pay on 
your home. They provide fire and police protection. Local government regu- 
lates how fauildinns must be constructed and where they must locate. They take 
care of roads, water, and sewerage" supp I ies. County governments usually pro- 
vide recreation and health services for citizens. By speaking at public hear- 
ings conducted by the County Commission, you can have an impact on many 
decisions that really affect you where you live. 

As a student, you may be more aware df the role that your County Board of 
Education plays In your life. The School Board, composed of elected members, 
sets policies concerning hiring of faculty and funding various programs. _ The 
Board of Education must'make sure that school buildings are in good physical 
condition. The members of the board make sure that day-to-day operations like 
bus service and school lunches are In order. If they have particular concerns 
about classes or curriculum, the Board members will discuss them with the proper 
school officials. The Board of Education controls the school system's budget 
and must set a property tax lO make certain that sufficient funds are collected. 
Like fne County Commission, the Board of Education holds pubjic hearings so 
that Interested citizens can express their views. 

When you speak before the County Commission or the School r^oard, remember 
that the officiafs arc truly interested in listening to your views. But they 
cannot read your mind. You must speak clearly and In an understandable manner. 
Rernember also that a public hearing Is an occasion to air conflicting points of 
viow. You cannot expect the Commlsionors or Board Members or your fellow citi- 
zens to always agree with your opinion. So you must speak persuasively and 
give reasons to help them see tnlngs your way. 
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. Appisndix F (continued) 

J'ffrropUCTWN- you clearly state which. Item on- the agenda* you wish to 
dl5c)iss? 60 you attempt to capture the Gbmmlss loners' or Roqrd members'.*- 
.attentjpn? 

PURPOSE- Do you clearly state your position on the Issue? 

REASONS- Do you give reasons for your position? Do you support those rea- 
sons wfth facts/^examples^ or common sense? 

-0ReAfffi^T+9fi — 9o-yeti-fiut-70w^- ffi e ssag Q to fiether so that the Commissioners 



or Etoard members con see how one Idea follows from another. Is It clear which 
of your points are the most important? 

: — QQicmjQ£lS:=_DQ_YQu_anjilcipate why some people might be opposed to your ^ 

point of view? Do you show why those objections are mistaken? 

/X»4CLUSJ0N- Do you end your message. with a statement that will help the, 
Cbmmlssipners remember your point of view? Do you thank the Commissioners or 
-'--Board- members for~t-hei^r--at:tentJ.on? 

. LANGUAGE STYLE- Do you speak fluently, meaningfully, and without too much 
slang? 

ORAL EXPRESSION- Do you use your tone of voice to keep your listeners' 
attention and to show' your sincerity? 

SPEECH RATE AMD VOLUME- Can you be heard easily? Do you speak at a speed 
that can be fol lowed without effort? 

GESTURES- Do you use your face, posture, and hands to reinforce what you are 
saying In a natural fashion? Do you look the Commissioners or Board members in . 
the eye? 

Improving your skills i-n these areas will help you to become a more 
effective communicator In ail your encounters. Including public hearings. 
Sometimes people are fearful or timid of speaking in a public setting. But 
If you believe in what you are saying, and have given a little thought about 
how to make others believe It as well, then you owe It to yourself to speak 
up. In this way you can have influence on your community and on your life. 



APPENDIX G 



INStRUaTlbfJS to TEACHERS FOR ADMINISTERING THE PUBLIC HEARING 
ORAL COWMUN I CATI ON ASSESSMENT 



General Concerns— . - 

pur primary concern Is that we maintain consistency In administering 
thIS; test.' Everyone must be treated alike.' 

1. Use uniform Tnstructlons. 

2. Maintain consistent demeanor — f rlei>d I Iness, encouragement, verbal ar^d 
nonverbal feedback. 

3. Keep testing conditions constant including setting and amount of dis- 
traction. 

Setting Ud the Room 

L. Set three chairs -and a table at the front of the room for the 
''Board Members.^* 

.2. Set remainder of chairs facing the front of the reom with an aisle 
in the middle. 

3. Place the speaker 's'podi urn or table in the middle aisle even with the 
front row of the audience. (Speakers will be standinq with the aud- 
ience, speaking to the ''Board Members'*.) ^ " 

4. If possible, tape the microphone to the speakcr^s podium. 

5. Place the video camera at a slinht anqle (3/4 profile of speaker) 
so that speakers dp not have to* hook directly at the camera wnen 
addressing the "Board Members." 

6. Check tape recorder. Return counter to "000" at start of each page. 

Introducing and Running the Task > 

I. Base your Introductlor. on the following script: "A Couni|-y Boa^rd of, 
of Education makes important decisions that affect students, teachers, 
and all .the citizens of the County. How many of you have concerns 
about the physical condition of schools you have attended? Maybe the 
schools were crowded or. needed air conditioning. Some of you may have 
wished that better textbooks were available or that different tynns 
of classes were offered. Maybe you have heard your parents discuss- 
ing the taxes on your home. A (bounty E5oard of Education has responsi- 
bility for these matters that have an Impact on the day-to-day functloi 
of the schools Ctid on the future of the* school system. 



r Append Tx' (5' (contlhuod) * 

:Now>tKe''Boc 1 '^^^ t*^nt their. dbcfslons aire Important to 

vdT?' . residents of the County-> so the ^Board welcomes, .people to Its^ 

meetings. At pub.lfc hearinns* any interested citlzea-'^students. 
parents, teachers—fi^ts a chance to express their view? to the- 
Board Members, ff people cone- to these meetlngsr and. tej I the School 
^A^ard Members what Is on their minds, then members of the community 
may have a great deal of Influence on thair schools. / ^ 

Today are going to practice what It would be like to speak at 
a public hearing before a"C6unty Hoard of Education. The Board hgs 
an agenda of Issues it will be. considering at this meeting. Each 
of you will choose one Issue about whfch you would like to make a, 
statement. You will have a few mlntites to think about what you want 
/to say to convince the Board "Members of your point of view. Jot 
down a few. notes to help you remember, but dpnH. try, to comp6se an 
.entire spe'^ech. Then each bf yoy will have a chanCe td speak to the' 
Board Members. 

I know that some people get a little nervous .about speaking out, and 
I know that some people get a little silly, too. But T hope that 
youMi take this seriously aiid really thjnk about what you would ^ant 
. to say about these decisions that could rVal ly^af feet your lives. 
Just ignore the camera. There^s really no need to be nervous.' None 
of us is an expert speaker. WeVe all In the same^boat. ,WeMl all 
just be interested in hearing what you We got to say." 

2. Hand out the agenda,. Read it aloud. Summarize the throe proposals. 
Ask If they are clear. (In answering questions try to redirect them 
to the studepTt: "YouMI have to think about what that means to you." 

3. Choose three students at random to take the "Bqard Member" seats. 

4. Give students three minutes to organize their thoughts^- 

5. Students should be called up in a random orderr They should speak 
to the "Board t^embers," not the teacher, the camera, or the cl&ss. 
Encourage (by example) students to applaud after each speaker. 

6> Record pertinent Information (students names, propbsa h number, etc.) 
on the "Videotape Recording Record." 

?• OonH forget to give the *'Board Members their turn at speaking. They 
should be the last to speak, addressing the remaining two seated 
"Board Members," • • 
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APPENDIX H 



Public Hearlnt] Feed « Back Fom 
Orlglna.l Draft' 



CLASS: 



DATE: 



SfcbBE: 



PERFORMANCE STANDARD: 



PROPOSAL 



SPEAK I NQ" ORDER 
RATER:' r 



■K INTRODUCTION: 



■ ( J ) none 

(2) Just narnes proposal 

■.{;3) names proposal and attempts to. capture Interest ' ? 

(4) names projaosal and provides novel or e^aborated aMroach 



■2. . PURPOSE: 



(I) no point of>vlew ' , ' 

' (2) vague point of view 

(3) unambiquously sta"tes position on proposal 

(4) 'states position with emphasis or situational, qualifier 



3, REASOMS: 



C 



(1) unsupported assertion 

(2) ynelaborated reasons given 

(3) at least one reason supported 

(4) -several reasons supported or especially apt support 



4, ORGAfJIZATIONr 



(1) Ideasf wholly unrelated 

(2) Ideas* only Impllcrtl^ related 

. (3) logical sequence or simpfe transitions 

C4.) proper empbasts and expfriclt connections' between ideas 



5. OPJECThONSr 
1 



(I)^ does not acknowledge reservations ^ 
'(?)-* arknowTednes but does not refute reservations 

(3) rofutesvat least one reservation ' - 

(4) refutes several reservations or especially apt refutation 



6, •CONCLUSION: 



(1) no conclusion or merely states thjjf^ remarks are finished 

(2) Just' thanks Commlssfon or jjjnt restates position 

(3) r^^states position and offers thanks 

*^C4) summarizes or concludes momornbly and of fer^^thanks 
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T: . UNGIJAne STYLE: 




slanq or Incomprehensf ble on several occasions 
very vaque or dtstracting "ff tiers" or wrfttenManguane 
minfmally fluent, appropriate formality 
vlv?Tj~plTraslng, highly corcprehenslWe 



8. ORAU- EXPRESS I ON: 



(hi — monotone — 

(2) inappropriate or distracting Inf lection. oq several occasions or memorized f| 
'(3) conversational variation In inflection ^ 
(4) tone of voice expresses conviction or emphasis 



9^ SPEECH RATE AND VOLUME: 



(I) 
(2) 



(4) 

10. GESTURES': 



Inaudible 

rate too fast or too, sloW - distracting 
-speech rate and vol ume-rdo^not_-.straln— I i-stoners,- 



variation In rate or volume used lor added expressiveoess.^ 



r 



(1) distr'-:tlng mannerisms or posture 

(2) no eye contact with Commissioners 

(3) eye contact estab fished, comfortable posture 

(4) facial, body, or hand gestures for emphasis <or illustration 
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APPENDIX I 



STUDENT: 



Public Hearinn Feedback Form, Final Version 

^ CLASS: , . DATE: SPEAKING ORDER: 

PROPOSAL P: RATER: 



PERFORMANCF STANDARD: 



.1-. INTRODUCTION 

( I ) none 



(2) just nafnes proposal 
'('3) namefe proposal and ir^troduces self 
(4) names proposal and attorhpts to capture interest 



irroNT 



( 1 ) no point of view 

(2) distorted point of view as It rajates to the orop"bsal 

(3) vaque point of vlow - weak In statinq position 
X4)~uliambrglious1y states position on proposal 



shows confusion 



3. REASONS ; 

(i) 

(2) 
(3) 
(4) 



unsupported assertion 
unelaboraied reasons qlven 
at least one reason supported 

more than one reason supported or especially apt support 



4» ORGANIZATIOM; 



H) 
• (2) 
(3) 
(4) 

5. CONCLUSION:* 



Ideas wholly unrelated 

only one Idea expressed - no relationship possible 
more than one Idea but no logical sequence 
logical sequence of Ideas " ■ 



(1) ho conclusion or merely states that remarks are finished 

(2) Just thanks Commission or just restates position 

(3) restates position and offers thanks 

(4) summarizes position and offers thanks 



6. LANGUAGE STYLE 



(1) reads from notes or Is Incomprehensible 

(2) slanq or Inappropriate lanquaqe used 

(3) minimal ly f I uen; , approrlate formality 

(4) \ fluent, appropriate formality 



\ 




78 



..... ,. 



73: . 



^ ApBendlx I (continued) 
VOCAL DELIVERY: 



(I ) monotone or Inaudible 

(2( 'distractinn tone or rate 

(3) conversational tone and rate' 

(4) emphatf-tone,- varied' rate 



8. GESTURES: 



XLi_dl3±naciJ4W-m3nner4^ms--or--pos+ure 



(2) no eye contact with Commissioners 

(3) eye contact established, comfortable posture 

(4) extended eye contact and some appropriate ges'turlhg 
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' ■ APPENDIX J 

. * EMPLOYMENT I m*F.RV I EW FEEDBACK FORM 
^SfuOENf; - ' CLASS: . DATE: _ 



:5CbRE: , . PERFORMANCE STANDARD: RATEP- ^ - " " 

'SOCIAL RITUALS: 

'(|j)- falls to orefif, thank, or take leave of Interviewer 

(2) oreet^, thanks, takes leave In perfunctory or excessively ornate manner 

r -(3). jjreets, thanks, takes leave In sincere and appropriately polite manner • ^ 
(4)^ greeting states purpose of Interview, leave tanlng states hope for future 
Interaction 

RESPONSIVENESS:-" 

(1) fails to verbal I ze several responses 

(-2) single nhrase response to many open quastlons - ■ _ , 

(3) single phrase response to closed questions, multiple phrase response to open 

- questions , ^ , , 

(4) volunteers elaborate reply to closed questions anticipating Interviewer s 

Interest * 

INFORTIATIVENESS: # " ..... ^ /~ , , 

(1) replies with irrelevant Informat'lon 'or states lack of knowledge/opinion 

. (2) replies are very vanue'' 

(3) replies wVrh minimally adequate information 

(4) replies cop4ain self assessfoonts and specific examples adapted to interviewer 
perspective 

INITIATIVE:. 

(1) assumes a wholly passive role 

(2) questions or comments only when a&ked to do so . 

(1) vpTunteors a question or comment ~ ■ ' ■ ' 

(4) vp.|junteers a question or comment which demonstrates knowJ edge/competence 

INTERPERSONAL MANNER ' " - 

(|> hostile, overly familiar, or obsequious. , 
'(2) apathetic or unusual ly nervous 

(3) relaxed, conversational 

(4) confident, dynamic" 

LANGUAi5E STYLE: 

(1) slang or Incomprehensible on several occasions 

(2) , very vague or dlstractinn ''f 11 lers^' 

(3) minimal ly fluent, appropriate formality 

(4) ylyld phrasing, highly comprehensible 

ORAL 'express ION: ' 

(1) monotone . . 

(2) Iftapproprlate or dlstractinn inf.lectlon on several occasions 

(3) ' Dii^turiil variation In Inflection 

U) tone of voice expresses conviction or emphasis 



' ;SreECH flAT E WD VOUJME: '/ - 

(1) ln?5udlbte 

\ (2)* ^^rato' too f ast or too slow - distracting 
X!^) natural speech rate and volume 

(4) variation In rate or volume used for added expressiveness 

GESTURES: , • 
• ' (I) distracting mannerisms or posture . f 

(2) no eye- contact with interviewer • 

^ (-3) eye contact established, comfortable posture 

(4) facial, body, or hand gestures used for emphasis or illustration. 
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Date: 



of 



Tape No* 



VIDEOTAPE RECOSDINO RECORD - *Publ ic Hearing 
Oral Ass<»5Sf)e1it 



Period: 



Class: 



Test \Adml n i strator : 



School: «BHS 



. iSpeaklng 
.Order 



Tape • 
<bunter 



000 



student Mame 



Proposa I N6.<. 



\~ 



\5. 



,84 



, 9. 



10. 



t.i 



12. 



13. 



14: 



15. 



16., 



17. 



18. 



19, 
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rater 2 \ 



Oral 06r^^^ Assessment-- 
^, Raters * Ta 1 1 y Sheet 



, Rater #1 Rafer #2 
tanA ,V roiintor ,7 student name -total +/- cut-off total +/- cut -off - 
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total 


lo'ater ' 
+/- cut-off 
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APPENDIX M 



•GLYNN C0Uf4TY SCHOOLS 
SPEAK I N& COMPETENCY REPORT 
• ' 1980-8!- 
Tenth Grade 
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^ ^ : 1, partlcfpated In an activity deslgned^ 

* • -Name* pt- Student ' 
to test his/her abiJItv to speak durinc a public hearing. The results of 



;M •^i: ' -th t s pt:6st- a re/g i ven be 1 ow . 



The student achieved at least the nin imum [evel required for graduation^ 
'Effgrts should ine made^ hpwever, I'o continue development of these skills. 



T The student f^^ijed^ to achieve the fninimum level required for graduation^ 
The student nqeds to work very hard to improve these skills. 



Areas. in which improvement is needed the nost: 
/j^^^ introduction 

DOS? tion 
_ reasons 

organization 

cone ! us i on. 

lan(]6aae style 

vocal delivery 
gestures 



ERIC. 



